Adding a new project to the development pipeline.
A few months ago I developed two iterations through a system prototype that would spawn up a listener bot using an identity anonymizer from a unique-per-session connection point from a core controller and then connect to a list of IRC channels. From there these listeners would simply relay the content of the channels in realtime, line by line, to a central database.
There was also a little bootstrap frontend that when pointed to the database would display the content but was built in a way to prevent google indexing.
I’ve been thinking about how valuable the content in major development support channels would be if they were in fact google indexed and have decided that they are, in fact, useful to communities.
So, if someone’s searching, for, say, python reading material, I think this would be a good resource if #python on freenode were indexed by crawlers.
Since I enjoy python, I think that channel is a great candidate.
I have to admit for transparency that there are dual purposes served by this as I’ve for a long time wanted to build an IRC content aggregator that had built in identity research features (click a username, see the hostnames associated from previous comments, click one of those, see the usernames associated from previous comments– and even be able to generate a little diagram that spiders out).
With this project I’ll be separating out hosting from development, i.e., I will be building the system used, someone else will be hosting it as a showcase.
The project’s name is ircthulu, and has a project channel created called #ircthulu, also on freenode. The beta iteration was quite unpolished but did serve as an interesting POC. The first initial iteration is actually up on github here, with the second iteration being a total rewrite that was never released. The early prototype on github, as you can see is not ready to be released as an official product. It depends on too many outside components and makes too many configuration assumptions to be a serious publication system.
While the initial scope of this project goes far beyond #python on freenode, I need test data.
No disruptions will be made by the testing, it is quite silent, so this is actually good timing.
I think if properly managed in terms of time that the version I’ve been working on in between other projects could be beneficial to the open source community for a wide number of reasons, the most notable of which is transparency in operation style.
More updates over the coming year or so as I build out something worthy of releasing. I’ve got some more important projects along the way but this will definitely be getting some development time.