I reworked the consumer in data and have titled it synapse.
Performance Notes
Based on previous tests and the current behaviour of ircthulu’s tenta and data backend, I think we will not even need to consider scaling until we’re at about the 2000 simultaneous channels mark.
It’s there if we need it. I can cluster and autoscale it easily. Once we do that I seriously doubt if we could not get almost every major IRC channel on every major network logging simultaneously without significant data costs.
Stability
The Tenta layer is decentralized and distributed. The MQ is clustered. The DB is clustered. The synapse between them is rather butch in the multithreaded manner in which it is processing. I’m overall pretty confident in its ability to store the content without intervention at this point and am hardening the way in which it is bundled for distribution.
API and UI is next
From here it’s the API and the UI to actually serve the content, and just making sure the feeds are still coming in. Some are already active.
Participants
If you have a persistent connection and would like to run the tenta client as a systemd background service please reach out at punches.chris@gmail.com with details about the channel you would like to host.