Giter Club home page Giter Club logo

fs2's Introduction

FS2: Functional Streams for Scala

Continuous Integration Gitter Chat Latest version

Overview

FS2 is a library for purely functional, effectful, and polymorphic stream processing library in the Scala programming language. Its design goals are compositionality, expressiveness, resource safety, and speed. The name is a modified acronym for Functional Streams for Scala (FSS, or FS2).

FS2 is available for Scala 2.12, Scala 2.13, and Scala.js. FS2 is built upon two major functional libraries for Scala, Cats, and Cats-Effect. Regardless of those dependencies, FS2 core types (streams and pulls) are polymorphic in the effect type (as long as it is compatible with cats-effect typeclasses), and thus FS2 can be used with other effect libraries, such as Monix.

Prior to the 0.9 release in 2016, FS2 was known as scalaz-stream, which was based on the scalaz library.

Getting Started

Quick links:

Where to get the latest version

The latest version is 2.4.x. See the badge at the top of the README for the exact version number.

If upgrading from the 1.0 series, see the release notes for 2.0.0 for help with upgrading.

// available for 2.12, 2.13
libraryDependencies += "co.fs2" %% "fs2-core" % "2.4.0" // For cats 2 and cats-effect 2

// optional I/O library
libraryDependencies += "co.fs2" %% "fs2-io" % "2.4.0"

// optional reactive streams interop
libraryDependencies += "co.fs2" %% "fs2-reactive-streams" % "2.4.0"

// optional experimental library
libraryDependencies += "co.fs2" %% "fs2-experimental" % "2.4.0"

The fs2-core library is also supported on Scala.js:

libraryDependencies += "co.fs2" %%% "fs2-core" % "2.4.0"

There are detailed migration guides for migrating from older versions.

Example

FS2 is a streaming I/O library. The design goals are compositionality, expressiveness, resource safety, and speed. Here's a simple example of its use:

import cats.effect.{Blocker, ExitCode, IO, IOApp, Resource}
import cats.implicits._
import fs2.{io, text, Stream}
import java.nio.file.Paths

object Converter extends IOApp {

  val converter: Stream[IO, Unit] = Stream.resource(Blocker[IO]).flatMap { blocker =>
    def fahrenheitToCelsius(f: Double): Double =
      (f - 32.0) * (5.0/9.0)

    io.file.readAll[IO](Paths.get("testdata/fahrenheit.txt"), blocker, 4096)
      .through(text.utf8Decode)
      .through(text.lines)
      .filter(s => !s.trim.isEmpty && !s.startsWith("//"))
      .map(line => fahrenheitToCelsius(line.toDouble).toString)
      .intersperse("\n")
      .through(text.utf8Encode)
      .through(io.file.writeAll(Paths.get("testdata/celsius.txt"), blocker))
  }

  def run(args: List[String]): IO[ExitCode] =
    converter.compile.drain.as(ExitCode.Success)
}

This will construct a program that reads lines incrementally from testdata/fahrenheit.txt, skipping blank lines and commented lines. It then parses temperatures in degrees Fahrenheit, converts these to Celsius, UTF-8 encodes the output, and writes incrementally to testdata/celsius.txt, using constant memory. The input and output files will be closed upon normal termination or if exceptions occur.

Note that this example is specialised to IO for simplicity, but Stream is fully polymorphic in the effect type (the F[_] in Stream[F, A]), as long as F[_] is compatible with the cats-effect typeclasses.

The library supports a number of other interesting use cases:

  • Zipping and merging of streams: A streaming computation may read from multiple sources in a streaming fashion, zipping or merging their elements using an arbitrary function. In general, clients have a great deal of flexibility in what sort of topologies they can define, due to Stream being a first class entity with a very rich algebra of combinators.
  • Dynamic resource allocation: A streaming computation may allocate resources dynamically (for instance, reading a list of files to process from a stream built off a network socket), and the library will ensure these resources get released upon normal termination or if exceptions occur.
  • Nondeterministic and concurrent processing: A computation may read from multiple input streams simultaneously, using whichever result comes back first, and a pipeline of transformations can allow for nondeterminism and queueing at each stage. Due to several concurrency combinators and data structures, streams can be used as light-weight, declarative threads to build complex concurrent behaviour compositionally.

These features mean that FS2 goes beyond streaming I/O to offer a very general and declarative model for arbitrary control flow.

Documentation and getting help

  • There are Scaladoc API documentations for the core library, which defines and implements the core types for streams and pulls, as well as the type aliases for pipes and sinks. The io library provides FS2 bindings for NIO-based file I/O and TCP/UDP networking.
  • The official guide is a good starting point for learning more about the library.
  • The documentation page is intended to serve as a list of all references, including conference presentation recordings, academic papers, and blog posts, on the use and implementation of fs2.
  • The FAQ has frequently asked questions. Feel free to open issues or PRs with additions to the FAQ!
  • Also feel free to come discuss and ask/answer questions in the gitter channel and/or on StackOverflow using the tag FS2. Gitter will generally get you a quicker answer.

Projects using FS2

You can find a list of libraries and integrations with data stores built on top of FS2 here: https://fs2.io/ecosystem.html.

If you have a project you'd like to include in this list, either open a PR or let us know in the gitter channel and we'll add a link to it.

Adopters

Here's a (non-exhaustive) list of companies that use FS2 in production. Don't see yours? You can add it in a PR!

Acknowledgments

YourKit

Special thanks to YourKit for supporting this project's ongoing performance tuning efforts with licenses to their excellent product.

Code of Conduct

See the Code of Conduct.

fs2's People

Contributors

mpilquist avatar pchiusano avatar pchlupacek avatar fthomas avatar systemfw avatar djspiewak avatar scala-steward avatar diesalbla avatar kubukoz avatar amarrella avatar fiadliel avatar alissapajer avatar adamchlupacek avatar yilinwei avatar christopherdavenport avatar rossabaker avatar tom91136 avatar gvolpe avatar zaneli avatar runarorama avatar cremboc avatar mgibowski avatar radekm avatar bryce-anderson avatar greg-a-atkinson avatar zainab-ali avatar johnynek avatar changlinli avatar shengc avatar aeons avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.