Giter Club home page Giter Club logo

go_iter's Introduction

go_iter

Go iter tools (for iterating , mapping, filtering, reducing streams -represented as channels-)

Thank you to Agustín Díaz for his contributions.

  • to install : go get -u github.com/serge-hulne/go_iter
  • Documentation : See in the Documentation directory.

Defines:

  • Filter
  • Map
  • Reduce
  • Range
  • Take.

Can easily be extended/generalized to all collection types.

See examples for more information:

Partial example (code snippet) :

In the example, data coming from an input channel are mapped/filtered, using Map(), to an output channel. Map.is one of the functions provided by go_iter.

	// Data
	nums := []int{0, 1, 2, 3, 4, 5}

	// Data -> iterator:
	generated := Iterable_from_Array(nums)

	// -> 0, 1, 2, 3, 4, 5

	// Mapping x -> 2 * x for all elements of the iterator:
	even := Map(generated, func(x int) int {
		return 2 * x
	})

	// -> 0, 2, 4, 6, 8, 10

	// Mapping all the elements of 'even' to it's square value:
	even_and_squared := Map(even, func(x int) int {
		return x * x
	})

	// -> 0, 4, 16, 36, 64, 100

	println("Final")

	// Displaying results:
	for item := range even_and_squared {
		fmt.Printf("%#v ", item)
	}

go_iter's People

Contributors

agustinramirodiaz avatar serge-hulne avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

go_iter's Issues

Hard to compose iterators together

I have some performance concerns regarding the use of channels as the only way to propagate, but there's not a nice way for you to address that without, like, rewriting the whole thing. So I'll complain about something that's straightforward enough:

currently, your worker interface takes two channels and returns nil. Instead, a worker should be expected to take a context and a receive-only channel, returning a new context and a new receive-only channel. This change would ensure that the responsibility for each channel is clear, error propagation on a pipeline is controlled at each step by the propagator, and the implementor is far less likely to leave a thread hanging in the background, still running.

Unfortunately, I'd still have the doesn't-know-how-to-share-memory issue, so I'd still be using native loops and/or chains of reader/writer interfaces for same-thread iteration.

If the parent function returns early, you end up with blocked orphaned go-routines

If the parent function returns early, you end up with blocked orphaned go-routines

In each case, you use a go-routine to populate the channel you are ranging over, and as you said, that blocks until the next element is read using the range function. Isn't this problematic if you do error handling in the for loop? Even though it won't block the main thread, wouldn't the go-routine block and stay around forever as long as the main thread runs? If this is a server, then as long as the server is alive? If it serves many requests, you might just keep adding more and more go-routines!

Example:

for item := range Range(10) {
	if item == 2 {
		return
	}
}

The previous example would return without reading the rest of the ints in the channel and thus leave the orphaned go-routine around.

Full example file that demonstrates the orphaned go-routines getting created:

package main

import (
	"fmt"
	"runtime"

	"github.com/lucemhealth/go-common/lucemtelemetry"
)

func main() {
	lucemtelemetry.ConfigureZerologLogger()

	for i := 0; i < 10; i++ {
		fmt.Printf("request number: %d\n", i)
		fmt.Printf("starting number of routines: %d\n", runtime.NumGoroutine())
		doForLoop()
		fmt.Printf("ending number of routines: %d\n", runtime.NumGoroutine())
		fmt.Println()
	}
}

func doForLoop() {
	for item := range Range(10) {
		if item == 2 {
			return
		}
	}
}

func Range(nmax int) chan int {
	out := make(chan int)
	go func() {
		defer close(out)
		for index := 0; index < nmax; index++ {
			out <- index
		}
	}()
	return out
}

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.