usefathom / fathom Goto Github PK
View Code? Open in Web Editor NEWFathom Lite. Simple, privacy-focused website analytics. Built with Golang & Preact.
Home Page: https://usefathom.com/
License: MIT License
Fathom Lite. Simple, privacy-focused website analytics. Built with Golang & Preact.
Home Page: https://usefathom.com/
License: MIT License
Not sure how but seeing pathnames without a leading slash come in. Let's validate this server side, just in case.
Right now, the premade time frames are:
These aren’t much help if you are early into a week / month / year. Wouldn’t it make more sense to have:
Alternatively (or additionally), it could also be interesting to overlay the previous instance of that time frame. So when you select this month, you can toggle a curve for last month as well. That adds some complexity though.
Either way, it would be nice to have easy access to relative time frames.
For some reason most of my pages are including my domain in the page name as shown below. That is causing broken links as the the pages don't actually exist at domain.com/domain.com/. Maybe has something to do with WordPress' trailing slash: https://anchor.host/blog/.
When changing the time frame (Today / This Week / This Month / This Year), the height of the graph area always increases a few pixels in height with every change. It pushes down the rest of the UI, too. Subtle, but noticeable if you click around. I don’t have real data, so not sure if that influences the behaviour.
Using Safari 11.1 on macOS.
I'm currently running an OpenWebAnalytics server, but would love to switch that over to ana! Any ideas if migration will ever be possible?
Because it's May 31, the date ranges started acting up.
Hi! I’m excited about Fathom. I have been looking for and thinking about something like this for some time. It’s a challenging project from a business standpoint, so I believe keeping it really simple and narrow in scope is key.
I love that design seems to be a focus for Fathom, and @pjrvs is doing fantastic work. I wanted to share some ideas around the UX that have been on my mind for a while. This is from the perspective of editorial sites likes blogs, which are probably the primary use case for Fathom, right?
So Fathom right now is built around the core metrics of visitors, top content and referrals. I think that’s spot on. It’s the holy trinity of what most people will ever need on an ongoing basis.
Yet, these metrics only matter over time. It’s about how they change. Are there more visitors than before? Does my updated Facebook page drive more traffic than the old one? When exactly did my blog post go viral? Time gives these metrics context and meaning, which results in insights.
The most straightforward way to surface that change over time is a proper graph. Simple up / down arrows are fine as a first indicator, but lack context. And sparklines don’t really give you the option to dive in deeper or interact with the data.
Here is a rough sketch of how the basic layout could work. You select a timeframe at the top, and the three sections below change accordingly. Maybe there could be a big visitor counter as part of the graph.
Now for the interesting part. I think the real power in these three data sets (visitors, content, referrals) is in their relation to each other. I want to know which of my blog posts was successful on Facebook, how a specific post did over a certain time frame, or where the visitors for that post came from. I want to explore these connections, and discover new insights on my own.
The simplicity of Fathom provides a great opportunity for an intuitive UI here. You could just click on any of the data points to filter all the other data for it. So if you click on a content page, you see the visitor graph and referrals for that page. If you click on a referral, you see the target pages and visitor graph for that referral. If you select a time frame on the graph, you see the content and referrals for that time frame.
It’s a super simple concept, but I think it’s powerful and intuitive for casual users.
Some quick notes on the metrics I left out:
Anyway, let me know what you think. I’d be happy to talk this through and develop it further if you think there could be something here. I know it’s still early days, so maybe I’m a bit overzealous here. But at least I got it out of my mind now. ;)
Thanks for working on this @dannyvankooten. Cheers!
I am not able to reach a (MariaDB) SQL Database, when setting the FATHOM_DATABASE_HOST
and couldn't determine from the wiki documentation what the FATHOM_DATABASE_HOST
expects exactly. I am seeing the error below. As you can see I am getting:
ERR panic: default addr for network 'galera-service.service.consul' unknown
2018-06-12 16:35:07 [APP/PROC/WEB/0] ERR time="2018-06-12T14:35:07Z" level=info msg="Fathom 1.0.0"
2018-06-12 16:35:07 [APP/PROC/WEB/0] ERR time="2018-06-12T14:35:07Z" level=warning msg="Missing configuration file. Using defaults."
2018-06-12 16:35:07 [APP/PROC/WEB/0] ERR panic: default addr for network 'galera-service.service.consul' unknown
2018-06-12 16:35:07 [APP/PROC/WEB/0] ERR panic: runtime error: invalid memory address or nil pointer dereference
2018-06-12 16:35:07 [APP/PROC/WEB/0] ERR [signal SIGSEGV: segmentation violation code=0x1 addr=0x18 pc=0x819aac]
2018-06-12 16:35:07 [APP/PROC/WEB/0] ERR goroutine 1 [running]:
2018-06-12 16:35:07 [APP/PROC/WEB/0] ERR main.after(0xc4200ca9a0, 0xc420079988, 0x57af44)
2018-06-12 16:35:07 [APP/PROC/WEB/0] ERR /go/src/github.com/usefathom/fathom/cmd/fathom/main.go:95 +0x2c
2018-06-12 16:35:07 [APP/PROC/WEB/0] ERR github.com/usefathom/fathom/vendor/github.com/urfave/cli.(*App).Run.func1(0xc420300000, 0xc4200ca9a0, 0xc420079d40)
2018-06-12 16:35:07 [APP/PROC/WEB/0] ERR /go/src/github.com/usefathom/fathom/vendor/github.com/urfave/cli/app.go:233 +0x3b
2018-06-12 16:35:07 [APP/PROC/WEB/0] ERR panic(0x9ccaa0, 0xc420184550)
2018-06-12 16:35:07 [APP/PROC/WEB/0] ERR /usr/local/go/src/runtime/panic.go:502 +0x229
2018-06-12 16:35:07 [APP/PROC/WEB/0] ERR github.com/usefathom/fathom/vendor/github.com/jmoiron/sqlx.MustConnect(0xc420026057, 0x5, 0xc420306000, 0x88, 0xa64b08)
2018-06-12 16:35:07 [APP/PROC/WEB/0] ERR /go/src/github.com/usefathom/fathom/vendor/github.com/jmoiron/sqlx/sqlx.go:652 +0x80
2018-06-12 16:35:07 [APP/PROC/WEB/0] ERR github.com/usefathom/fathom/pkg/datastore/sqlstore.New(0xc4200b8a00, 0x4)
2018-06-12 16:35:07 [APP/PROC/WEB/0] ERR /go/src/github.com/usefathom/fathom/pkg/datastore/sqlstore/sqlstore.go:26 +0x5d
2018-06-12 16:35:07 [APP/PROC/WEB/0] ERR github.com/usefathom/fathom/pkg/datastore.New(0xc4200b8a00, 0x4, 0xc42012f480)
2018-06-12 16:35:07 [APP/PROC/WEB/0] ERR /go/src/github.com/usefathom/fathom/pkg/datastore/datastore.go:59 +0x2b
2018-06-12 16:35:07 [APP/PROC/WEB/0] ERR main.before(0xc4200ca9a0, 0xb54ee8, 0xc420300000)
2018-06-12 16:35:07 [APP/PROC/WEB/0] ERR /go/src/github.com/usefathom/fathom/cmd/fathom/main.go:90 +0x93
2018-06-12 16:35:07 [APP/PROC/WEB/0] ERR github.com/usefathom/fathom/vendor/github.com/urfave/cli.(*App).Run(0xc420300000, 0xc4200bc000, 0x2, 0x2, 0x0, 0x0)
2018-06-12 16:35:07 [APP/PROC/WEB/0] ERR /go/src/github.com/usefathom/fathom/vendor/github.com/urfave/cli/app.go:244 +0x738
2018-06-12 16:35:07 [APP/PROC/WEB/0] ERR main.main()
2018-06-12 16:35:07 [APP/PROC/WEB/0] ERR /go/src/github.com/usefathom/fathom/cmd/fathom/main.go:80 +0x8c1
Any idea why I can't connect to that DB?
Loving the Fathom concept. Would it be possible to run a limited version on Heroku by any chance?
Cheers!
MySQL only allows keys for a maximum of 767 bytes, where with utf8mb4 each character can take up to 4 bytes meaning we can only create indices of columns up to 767/4=191
characters.
If we explicitly set the charset to utf8
then we should be able to use columns with 255 characters.
Alternatively, we can probably get away with smaller columns for certain pieces of data.
Web browsers automatically prefetch or even prerender web pages in the background on behalf of the user. The user may never see the page, so they should make sure to exclude them from analytics.
Chrome: ('visibilityState' in document && document.visibilityState=='prerender')
Firefox: X-Moz: prefetch
(HTTP request header)
Safari: X-Purpose: preview
(HTTP request header)
Edge: undetectable
Hello,
Thanks for this nice project. However, looks like I couldn't able to run make build
successfully.
Here is the issue I get:
[20:19:09] Using gulpfile ~/programs/go/fathom/gulpfile.js
[20:19:09] Starting 'browserify'...
[20:19:09] Starting 'sass'...
[20:19:09] Starting 'html'...
[20:19:09] Starting 'img'...
[20:19:09] Starting 'minify'...
[20:19:15] Finished 'html' after 5.45 s
[20:19:15] Finished 'minify' after 5.45 s
[20:19:15] Finished 'img' after 5.46 s
[20:19:15] Finished 'sass' after 5.8 s
[20:19:17] Finished 'browserify' after 7.65 s
[20:19:17] Starting 'default'...
[20:19:17] Finished 'default' after 56 μs
packr build -v -ldflags '-w -extldflags "-static"' -o fathom ./cmd/fathom
make: packr: No such file or directory
make: *** [fathom] Error 1
For when the header is omitted. Doesn't happen often, but starting to see it happen occasionally.
Alternatively, we could send this in the /collect
request by adding it client-side?
What do you say of an "official" automated Docker build for fathom?
In the README you should change the order. Because when I wanted to migrate the database, I got the error that there is no bin/migrate
in my GOPATH. After the installation of the dependencies it worked.
Disclaimer: I’m not a lawyer and this isn’t legal advise.
It would be useful for General Data Protection Regulation (GDPR) compliance to not store IP addresses, cookie identifiers, or other unique fingerprints. The current unique identifiers can be decoded back to IP addresses. See EU GDPR and personal data in web server logs for context.
I’d like to see this as the default mode, but at least make it an option. This could be a unique selling point
IP addresses also aren’t all that useful any more for assigning a unique identifier as mobile devices roam between different networks several times in a normal day (at home, mobile carrier, work, café, etc.) IPv6 reduces the usefulness of this further by assigning new addresses periodically (usually once per reboot, reconnect: or 48, 24, or 12 hours depending on operating system and network environment).
Here are some ideas and alternatives approaches to get the same data in aggregate without assigning unique identifiers to each user:
Pageviews per session and number of unique users:
/collect
that runs an incremental short-lived/session pageview counters. E.g. Set-Cookie: ana_pageviews=1; path=/collect; max-age=3600
(1 hour session). Second request you send back the same cookie with a value of 2, etc.User-retention/repeat visitors:
/collect
that includes an imprecise timestamp (e.g. only daily precision to avoid them being too unique). E.g. Set-Cookie: ana_lastvisit=2018-04-23; path=/collect; max-age=7776000
(3 months). Reset on every visit.now() - $cookie['ana_lastvisit']
. Maybe don’t track this within an active session ($cookie['ana_pageviews']
is set)?What else is needed to track?
On the use of cookies: The cookies are transparent (even self-explanatory), their use is easy to explain in a privacy policy, and in my opinion they should be GDPR-friendly. They’re not used to track the behaviour of an individual users, just the movements and trends in the herd.
Disclaimer: I’m not a lawyer and this isn’t legal advise.
Bounce = any single pageview that did not result in any additional pageviews/events.
Related to #56
Basically we should limit migration files to one-query per file.
And in a later version, perhaps allow a "timezone" setting on a per-user basis?
To limit the number of INSERT
queries for super high traffic sites.
I would be cool if ana works with different databases. To have some abstraction and some kind of models I would suggest the usage of github.com/russross/meddler. If you also want to have an idea how to use it in the wild I would suggest to take a look at https://github.com/drone/drone/tree/master/store/datastore which is using it quite long and also with bigger load.
suggestion:
How about add support for multiple site (domain), so I can select to see which site's data.
And in backend, filter by hostname in pageviews table.
same as #36
Hi, I am trying out fathom today and am feeding my fathom instance from 4 different sources. One of them is regularly accessed by a headless browser to perform tests. In the fathom dashboard I can see multiple page loads for that instance, but it specifies 0 unique visitors.
Am I correct to assume that fathom somehow tries to detect potential non-human visitors and doesn't count them or is this just a bug? It would be great if my headless access wasn't considered as a visitor, but shouldn't then the page loads also be at 0 ?
Thanks!
So we can run fathom
from anywhere on the FS.
I would like to use Fathom in an environment where disk storage isn't permanent, and using an SQL database would cost too much while a great NoSQL datastore is already available. I would like to implement the relevant parts in Fathom so that others in a similar situation can use it.
Do you think this could be doable in some way? I checked the code just today, only briefly, and the queries look rather straightforward so a transformation doesn't look impossible, but I am not sure.
@metalmatze pointed out why here. :)
Why? This is a common practice to just vendor all Go dependencies in vendor/ and put them into git too. I know you can currently fetch all dependencies that are specified in vendor/vendor.json yet you won't be able to, once someone decides to simply delete their repository. Hence you should just put them in there, otherwise you might not be able to built ana tomorrow anymore.
In order for this to be useable in the real world, Fathom needs to run over https. How about bundling in Let's Encrypt support. I'm not as familiar with Golang however it looks like some other projects might easily be plugged in like this one: https://github.com/gin-gonic/autotls.
From the command line I could see it being used something like this.
fathom server --webroot=./build --domain=<domain>
If the domain flag is used then it could automatically setup Let's Encrypt and redirect to HTTPS for that particular domain.
This was briefly talked about in #36 however I'd like to add some of my thoughts. Tracking multiple websites per Fathom instance would be great. Due to the complexities to site configuring (subdomains, subfolder sites, etc) I propose including an optional unique code which can be added to the tracking code.
This could function similar to Google's UA code and would allow for a clear way to parse through the data when viewing reports (per unique code). This would require some spam considering. Maybe each unique code could be assigned to a particular domain. Or even simpler, just give a way to parse the domain(s) you want when viewing the reports per each unique code. Like only show me unique code (XX-12345) from my site https://anchor.host and ignore the rest (like spammers who ninja tracking codes).
Possible use cases for unique codes include:
First of all, thanks to all contributors for this project!
I wanted to install Fathom on my server, but I'm getting a panic on start (see log below) when using MySQL. Leaving the defaults to SQLite works.
I first tried with the release and then to build it myself, but I get the same results.
Then I changed FATHOM_DATABASE_HOST
to 127.0.0.1
or the local IP, but no changes.
Here's my config (with changed user/pass):
FATHOM_DEBUG=true
FATHOM_SERVER_ADDR=9050
FATHOM_DATABASE_DRIVER="mysql"
FATHOM_DATABASE_NAME="fathom"
FATHOM_DATABASE_USER="fathom"
FATHOM_DATABASE_PASSWORD="NotAPassword"
FATHOM_DATABASE_HOST="localhost"
FATHOM_SECRET="NotASecret"
Here's the log:
root@Web:~# fathom --config=/root/fathom.env server
INFO[0000] Fathom 1.0.0
INFO[0000] Configuration file: /root/fathom.env
panic: default addr for network 'localhost' unknown
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x18 pc=0x81bf5c]
goroutine 1 [running]:
main.after(0xc4200c4b00, 0xc420071988, 0x57e574)
/go/src/github.com/usefathom/fathom/cmd/fathom/main.go:95 +0x2c
github.com/usefathom/fathom/vendor/github.com/urfave/cli.(*App).Run.func1(0xc420180000, 0xc4200c4b00, 0xc420071d40)
/go/src/github.com/usefathom/fathom/vendor/github.com/urfave/cli/app.go:233 +0x3b
panic(0x9c7ee0, 0xc42017c620)
/usr/local/go/src/runtime/panic.go:502 +0x229
github.com/usefathom/fathom/vendor/github.com/jmoiron/sqlx.MustConnect(0xc4200de7d7, 0x5, 0xc4200ce200, 0x39, 0x7ffc9bb2de10)
/go/src/github.com/usefathom/fathom/vendor/github.com/jmoiron/sqlx/sqlx.go:652 +0x80
github.com/usefathom/fathom/pkg/datastore/sqlstore.New(0xc4200b0aa0, 0xc42017c1e0)
/go/src/github.com/usefathom/fathom/pkg/datastore/sqlstore/sqlstore.go:26 +0x5d
github.com/usefathom/fathom/pkg/datastore.New(0xc4200b0aa0, 0x10, 0xc42013f240)
/go/src/github.com/usefathom/fathom/pkg/datastore/datastore.go:59 +0x2b
main.before(0xc4200c4b00, 0xa81b40, 0xc420180000)
/go/src/github.com/usefathom/fathom/cmd/fathom/main.go:90 +0x93
github.com/usefathom/fathom/vendor/github.com/urfave/cli.(*App).Run(0xc420180000, 0xc4200a4060, 0x3, 0x3, 0x0, 0x0)
/go/src/github.com/usefathom/fathom/vendor/github.com/urfave/cli/app.go:244 +0x738
main.main()
/go/src/github.com/usefathom/fathom/cmd/fathom/main.go:80 +0x8c1
During the installation process I had to npm install
and create the folders /stroage/sessions
, maybe that should be added to the installation process / readme.
The default flags package does not allow for much flexibilities and requires to globally register all accepted flags.
There should be a more specific package that allows fur subcommands with command-specific arguments & options (on top of global options like maybe --verbose
etc).
Hello,
Thanks for this nice library. I was planning to create one from scratch (for learning purposes) and found this. Kudos.. I just wanted to check if there is a way for us to capture the activity of the user (and save it as a video)? IBM TeaLeaf does that and its very powerful too at times for debugging. So just wonders, if that can be achieved here as well.
Affected queries:
It's just an idea, I love this project.
See: https://github.com/snowplow/referer-parser/tree/master/go
Steps to reproduce:
git clone fathom-repo
ls /path/to/fathom
docker build -t fathom .
--> throws:
Step 4/10 : RUN go get -u github.com/gobuffalo/packr/... && make docker
---> Running in a787be7b7c11
if [ ! -d "node_modules" ]; then npm install; fi
/bin/sh: 1: npm: not found
Makefile:38: recipe for target 'assets/build' failed
make: *** [assets/build] Error 127
The command '/bin/sh -c go get -u github.com/gobuffalo/packr/... && make docker' returned a non-zero code: 2
(Docker version: 18.03.1-ce-mac65 (24312))
In the sql import file you alter the table pageviews
and reference to the pages
table before it is created, that will result in an error during the import. Create the table pages
first, then alter pageviews
.
Hi,
is there a way to disable the tracking cookie completely? I’m not really interested in the returning visitor metric and would prefer to avoid tracking users completely.
Generally, with privacy being a focus for Fathom, it might be nice to have a list (or even a choice) of what exactly is tracked, how it is tracked, and for how long it is stored.
Cheers!
So low-traffic days aren't weighted the same as high-traffic days. Is especially important when website traffic is't evenly spread out.
Not sure about the best way to suggest edits to the readme and Wiki, I hope this is fine. Just some small things I stumbled upon when setting up Fathom for the first time:
replace yourfathom.com with the URL to your Fathom instance (twice).
There is only one instance.Other than that, set up was a breeze. I went from creating the server to running Fathom in less than 5 minutes. 👍
Return JSON on /api
endpoints, pretty user error page otherwise.
Hello!
I recently stumbled upon this project after reading a twitter thread about server logs analytics and other alternatives to Google Analytics...
I know it's only in its early stages, but I was wondering if there was potential support for multiple domains in a hidden roadmap?
I believe that right now the datastore and models are not relying on this type of entity yet, but what are your thoughts on this?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.