rinusser / hopgoblin Goto Github PK

Currently relative paths are resolved at compile time, resulting in binaries that won't work on other systems with different directory structures.

Change this behavior: paths should be resolved at the time of actually executing the binary, wherever it may reside. Probably update the path utils API too: look up a base directory and resolve everything relative to that.

Currently all HTTP requests and responses are read in their entirety, and only then handled as required. This works fine for small requests but e.g. causes timeouts with large downloads.

Change the handling of HTTP bodies to streaming I/O: anything up to and including HTTP headers should be handled as before, but request/response bodies (especially larger ones) should be read and written in blocks.

Currently a few hard-coded defaults can be changed from within application.ini. There are still a few values that require code changes to modify, at least the server listening port.

Find such settings and make them configurable in application.ini.

Make sure not to introduce import loops. If adding a particular configuration setting requires splitting up another package to work around Go's limitations, skip it.

There are 2 issues with certificates:

• if no default certificate is available both the main executable and the tests break.
• CA certificate loading isn't documented

Fix both by:

• making the default certificate optional: if none is available, disable https support and go on without it
• improve documentation: anything resources/certs/CA-* gets added to cert pool automatically

Currently site handlers are required to do their own caching, if any.

See what can be done to improve that. Find efficient ways of storing arbitrary blobs of data and matching its metadata. See what metadata is required for practical purposes. If there's a useful way that actually brings performance improvements warranting the additional complexity, implement it.

Currently log settings must be either hard-coded or passed in the command line.

Add log settings to the configuration file.

This will need a new way of handling custom config keys, probably add a method to fetch key/value pairs by key prefix.

Currently remote TLS certificate verification is disabled, as some sites failed verification.

Enable certificate verification, maybe add a per-handler or per-host setting. For sites that still fail, look into why exactly and see if it can be fixed.

Currently the installation via go get works, but could use some improvements:

• go get is complaining about the lack of .go files in the root package - add a dummy file, a doc.go or something
• SSL certificates aren't included - there'll be a ticket about making this optional later, but right now users should know they'll need to add cert.pem and key.pem files
• command paths are relative to the package root, explain that

Currently the built-in example site handler works, but doesn't show off a lot of possibilities.

Add more features, turn the handler into a showcase. Add at least interception, content mangling and analysis.

Currently only handful of HTTP response codes (200 OK, 404 Not Found and so on) are implemented, but a few commonly used ones are missing - most notably the 301/302 redirects. Response objects with unhandled codes can't be processed directly, requiring custom handler code for otherwise straightforward HTTP responses.

Add the HTTP response codes that are usually encountered in the wild, at least the redirects.

Currently the outgoing HTTPS port is hard-coded to 443, regardless of what port was actually requested from the client.

Change this: preserve the requested port and use it for outgoing connections.

Currently errors are triggered if e.g. browser tabs are closed during data transfers.

Regular browsing shouldn't trigger error messages. Find as many error types as possible and reduce their severity to DEBUG. See if such cases can be set up and tested for reliably in unit tests.

Currently function, type and property names used mixed naming schemes.

Find a useful naming scheme that fits Go's go-lorious visibility rules and apply it. Check again what a leading underscore means, maybe it can be used to mark methods/types intended as private.

Currently all outgoing requests are forwarded to an upstream proxy server.

Make this proxy server optional: http.Client should be able to send requests to servers directly. The proxy settings in application.ini should indicate whether to use the proxy.

Currently debugging/profiling builds require manual code changes to stop the application after e.g. a minute.

Add an application lifetime configuration setting and command-line argument, defaulting to endless operation.

Also add a logged countdown in incremental steps, maybe in a 5,2,1 logarithmic scale with the last minute and last seconds more verbose.

Currently each request and response is sent over separate connections, closing the connection afterwards.

Add support for connection keepalive to upstream servers and proxy keepalive to the client.

Currently server ports used in tests are hard-coded. This works as long as the ports are available, but breaks if e.g. another instance of the application is running.

Change the server and proxy listener code so arbitrary ports are opened and reported back to the invokers.