Edited 20180121
Staying aware of what's going on around you in a large software project is a bit of a chore. Meetings, reading various information sources like mail lists and wikis, maintaining queries and watches in support systems like gerrit, and so on. It would be nice if this could be automated, at least to some extent, so I gave it a try.
Roboreader continuously fetches new texts from various sources, reads them, and shows you only the texts that you are interested in. Currently it supports two sources, imap (email) and gerrit, but it's easy to add more sources. To figure out which texts that are of interest to you there are four match engines.
Pattern search
This match engine just applies a number of regular expressions you have created to the texts. If a text matches any of the regular expressions it is considered interesting.
Word frequency
This match engine figures out your interests by looking at which words you overuse. It looks at all the mails, patches, bug reports, etc that you have written and compares them to all the mails, patches, bug reports, etc that other people have written. If you overuse for example "anti-banding" it will assume you are interested in that topic and all texts containing "anti-banding" will be marked as interesting to you.
ANN
This match engine uses an ANN (artificial neural network) classifier to classify texts as "interesting" or "not interesting". The training data for the "interesting" class are the texts you have authored and the training data for the "not interesting" class are the texts that other people have authored. During training the ANN will learn patterns that are unique to you and use them to figure out which texts you might be interested in. Much like the "word frequency" match engine, but patterns instead of words, which makes this match engine a lot more flexible.
Carrot2
This match engine uses the open-source text clustering engine Carrot2. It clusters all new texts and all the texts you have authored. If a new text ends up in the same cluster as one of the texts you have authored it is considered interesting.
I am currently not working with large scale software development so I couldn't test how well it all worked myself, instead I set it loose on AOSP's (Android) gerrit and told it I was "Jakub Pawlowski", a software engineer at Google working with bluetooth. This approach is a bit flawed. All the texts are patches, which is unlikely in a real scenario. It is also hard for me to assess whether the found texts are of interest to Jakub or not as I'm not Jakub. Better than nothing though.
The results where a bit so-so. Pattern search works as expected but since you have to write the regular expressions yourself it doesn't add much value. Word frequency and ANN find interesting texts without too much noise, but I've got a feeling they miss a lot of interesting texts. I was unable to get anything useful out of Carrot2, way too much noise or no matches at all, could be a configuration issue though.
If you want to try it out yourself you can download the script here. To get going just edit config.xml and run the script. Current directory is used when looking for or storing files (like configuration and state).
python3 roboreader.py
No comments:
Post a Comment