Potential segfault in localtime_r invocations

Impact

Unix-like operating systems may segfault due to dereferencing a dangling pointer in specific circumstances. This requires an environment variable to be set in a different thread than the affected functions. This may occur without the user's knowledge, notably in a third-party library.

Workarounds

No workarounds are known.

References

• time-rs/time#293

See advisory page for additional details.

Currently in our build.rs we extract function documentation for tremor-query, mainly aggregation function documentation, from the tremor-www-docs repo pulled in via git submodule.

We have the aggregation docs in tremor-script already and can use those instead.

Version from cates.io doesnt compile with this error:

 Compiling lalrpop v0.18.1
error[E0308]: mismatched types
   --> /home/username/.cargo/registry/src/github.com-1ecc6299db9ec823/simd-json-0.3.3/src/lib.rs:213:86
    |
213 | fn please_compile_with_a_simd_compatible_cpu_setting_read_the_simdjonsrs_readme() -> ! {}
    |    ----------------------------------------------------------------------------      ^ expected `!`, found `()`
    |    |
    |    implicitly returns `()` as its body has no tail or `return` expression
    |
    = note:   expected type `!`
            found unit type `()`

In git version there is lalrpop v0.17.2 and everything works.

In the overall list of possible function completions for tremor-script, param and return type is not included currently. It used to work pre 0.8 (since we used to parse the doc markdown files where that info was structured in a certain way).

Once there’s support for encoding this info in a structured manner as part of tremor function doc, we can modify the language server to use it.

net2 crate has been deprecated; use socket2 instead

The net2 crate has been deprecated
and users are encouraged to considered socket2 instead.

See advisory page for additional details.

During development, while editing some trickle code, in a state where stuff doesnt compile i get multiple panics from the language server v0.10.0 like this:

thread 'main' panicked at 'index out of bounds: the len is 2 but the index is 18446744073709551615', src/lsp_utils.rs:59:55
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Data race when sending and receiving after closing a oneshot channel

If a tokio::sync::oneshot channel is closed (via the
oneshot::Receiver::close method), a data race may occur if the
oneshot::Sender::send method is called while the corresponding
oneshot::Receiver is awaited or calling try_recv.

When these methods are called concurrently on a closed channel, the two halves
of the channel can concurrently access a shared memory location, resulting in a
data race. This has been observed to cause memory corruption.

Note that the race only occurs when both halves of the channel are used
after the Receiver half has called close. Code where close is not used, or where the
Receiver is not awaited and try_recv is not called after calling close,
is not affected.

See tokio#4225 for more details.

See advisory page for additional details.

ansi_term is Unmaintained

The maintainer has adviced this crate is deprecated and will not
receive any maintenance.

The crate does not seem to have much dependencies and may or may not be ok to use as-is.

Last release seems to have been three years ago.

Possible Alternative(s)

The below list has not been vetted in any way and may or may not contain alternatives;

• anstyle
• nu-ansi-term
• yansi

See advisory page for additional details.

beef::Cow lacks a Sync bound on its Send trait allowing for data races

Affected versions of this crate did not have a T: Sync bound in the Send impl for Cow<'_, T, U>. This allows users to create data races by making Cow contain types that are (Send && !Sync) like Cell<_> or RefCell<_>.

Such data races can lead to memory corruption.

The flaw was corrected in commit d1c7658 by adding trait bounds T: Sync and T::Owned: Send to the Send impl for Cow<'_, T, U>.

See advisory page for additional details.

rental is unmaintained, author has moved on

The author encourages users to explore other solutions, or maintain a fork.

See advisory page for additional details.

Potential segfault in the time crate

Impact

Unix-like operating systems may segfault due to dereferencing a dangling pointer in specific circumstances. This requires an environment variable to be set in a different thread than the affected functions. This may occur without the user's knowledge, notably in a third-party library.

The affected functions from time 0.2.7 through 0.2.22 are:

• time::UtcOffset::local_offset_at
• time::UtcOffset::try_local_offset_at
• time::UtcOffset::current_local_offset
• time::UtcOffset::try_current_local_offset
• time::OffsetDateTime::now_local
• time::OffsetDateTime::try_now_local

The affected functions in time 0.1 (all versions) are:

• at
• at_utc

Non-Unix targets (including Windows and wasm) are unaffected.

Patches

Pending a proper fix, the internal method that determines the local offset has been modified to always return None on the affected operating systems. This has the effect of returning an Err on the try_* methods and UTC on the non-try_* methods.

Users and library authors with time in their dependency tree should perform cargo update, which will pull in the updated, unaffected code.

Users of time 0.1 do not have a patch and should upgrade to an unaffected version: time 0.2.23 or greater or the 0.3. series.

Workarounds

No workarounds are known.

References

time-rs/time#293

See advisory page for additional details.

Seeing following panic during tremor-language-server startup on windows:

C:\Users\juju\TREMOR>set RUST_BACKTRACE=full

C:\Users\juju\TREMOR>tremor-language-server --version
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 3, kind: NotFound, message: "The system cannot find the path specified." }', src\libcore\result.rs:1188:5
stack backtrace:
   0:     0x7ff603958a19 - onig_get_regex_by_callout_args
   1:     0x7ff603970cdb - onig_get_regex_by_callout_args
   2:     0x7ff603955294 - onig_get_regex_by_callout_args
   3:     0x7ff60395b4f9 - onig_get_regex_by_callout_args
   4:     0x7ff60395b14c - onig_get_regex_by_callout_args
   5:     0x7ff60395bd2c - onig_get_regex_by_callout_args
   6:     0x7ff60395b8b0 - onig_get_regex_by_callout_args
   7:     0x7ff60396d20d - onig_get_regex_by_callout_args
   8:     0x7ff60396d30f - onig_get_regex_by_callout_args
   9:     0x7ff6034fbc21 - <unknown>
  10:     0x7ff603537529 - <unknown>
  11:     0x7ff60349ba91 - <unknown>
  12:     0x7ff603513bc9 - <unknown>
  13:     0x7ff6034b8a20 - <unknown>
  14:     0x7ff60350d346 - <unknown>
  15:     0x7ff60395b707 - onig_get_regex_by_callout_args
  16:     0x7ff60395fd72 - onig_get_regex_by_callout_args
  17:     0x7ff60395bf52 - onig_get_regex_by_callout_args
  18:     0x7ff6034b8bd7 - <unknown>
  19:     0x7ff6039a47b4 - onig_unicode_define_user_property
  20:     0x7fff856b7bd4 - BaseThreadInitThunk
  21:     0x7fff85f6ce51 - RtlUserThreadStart

Tested with v0.7.4 of the server (via cargo install tremor-language-server), on rustc 1.41.1 (f3e1a954d 2020-02-24), windows 10.

The server does not error out on linux and mac environments. Fixing this will enable editors like vscode to use the server on windows too.

dirs is unmaintained, use dirs-next instead

The dirs crate is not maintained any more;
use dirs-next instead.

See advisory page for additional details.

term is looking for a new maintainer

The author of the term crate does not have time to maintain it and is looking
for a new maintainer.

Some maintained alternatives you can potentially switch to instead, depending
on your needs:

• crossterm
• termcolor
• yansi

See advisory page for additional details.

Links in archive can create arbitrary directories

When unpacking a tarball that contains a symlink the tar crate may create
directories outside of the directory it's supposed to unpack into.

The function errors when it's trying to create a file, but the folders are
already created at this point.

use std::{io, io::Result};
use tar::{Archive, Builder, EntryType, Header};

fn main() -> Result<()> {
    let mut buf = Vec::new();

    {
        let mut builder = Builder::new(&mut buf);

        // symlink: parent -> ..
        let mut header = Header::new_gnu();
        header.set_path("symlink")?;
        header.set_link_name("..")?;
        header.set_entry_type(EntryType::Symlink);
        header.set_size(0);
        header.set_cksum();
        builder.append(&header, io::empty())?;

        // file: symlink/exploit/foo/bar
        let mut header = Header::new_gnu();
        header.set_path("symlink/exploit/foo/bar")?;
        header.set_size(0);
        header.set_cksum();
        builder.append(&header, io::empty())?;

        builder.finish()?;
    };

    Archive::new(&*buf).unpack("demo")
}

This issue was discovered and reported by Martin Michaelis (@mgjm).

See advisory page for additional details.

Author of tower-lsp here! I just wanted to let you know that tower-lsp has been rapidly improving over the past several months and has seen a great many improvements since 0.4.1. Among other things, it has gained:

• Support for std::future::Future, the newest tokio, and async/await.
• Support for a crap-ton more LSP requests, almost supporting all of LSP 3.15.
• Support for all server-to-client requests, renaming Printer to Client.
• LanguageServer trait methods are no longer required, except for initialize and shutdown.
• Notification trait methods are now async fn.
• You can now call .await on serve() directly, eliminating the need for ExitReceiver.
• Backpressure is now handled properly.

If you would like to port your server to async/await and add some new features to your language server, such as goto declaration, goto definition, goto type definition, code formatting, document symbols, references, etc. 0.9.0 has just been published to Crates.io.

Not exactly sure if this announcement requires an official library RFC as per the contributing guidelines, but if it does, I do sincerely apologize for submitting this suggestion out of form. Feel free to close this issue if that is the case!

Error message:

Compiling tremor-language-server v0.10.1
error: failed to run custom build command for `tremor-language-server v0.10.1`

Caused by:
  process didn't exit successfully: `/var/folders/84/zhf78lt50hsghfh5s6g9jmk00000gn/T/cargo-installweU1XI/release/build/tremor-language-server-cf2b09233f27bfff/build-script-build` (exit code: 101)
  --- stdout
  Dumping function docs for tremor-script to: /var/folders/84/zhf78lt50hsghfh5s6g9jmk00000gn/T/cargo-installweU1XI/release/build/tremor-language-server-11042796ebe3999c/out/function_docs.tremor-script.bin
  Detected tremor-script version from cargo lock file: 0.10.1
  Could not find tremor-script src in local cargo registry, so downloading it now...
  Downloading crate `tremor-script==0.10.1` from https://crates.io/api/v1/crates/tremor-script/0.10.1/download

  --- stderr
  thread 'main' panicked at 'there is no reactor running, must be called from the context of a Tokio 1.x runtime', /Users/rohitdandamudi/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.2.0/src/runtime/blocking/pool.rs:85:33
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
warning: build failed, waiting for other jobs to finish...
error: failed to compile `tremor-language-server v0.10.1`, intermediate artifacts can be found at `/var/folders/84/zhf78lt50hsghfh5s6g9jmk00000gn/T/cargo-installweU1XI`

Caused by:
  build failed

OS: MacOS Mojave
Command: tremor-runtime rohitdandamudi$ cargo install tremor-language-server

This is more a tracking ticket - there is a RFC to integrate highlighting into the language server protocol - if this RFC is accepted we could re-use the highlighting we have in tremor-script instead of having to re-implement it in every editor. That would be a huge win I think.

So I'll drop a link to the upstream Issue on it here: microsoft/language-server-protocol#513

We do not currently build language server images ( docker, rpm, deb ) as part of the
release process for tremor. Adding these would mean folk have multiple options to
install the language server without having to install the rust toolchain and doing a build.

The images would be super useful in online IDE's such as github codespaces and gitpod.