When last I took to this blog, it was announcing that I’d be paying people to do freelance work on my EJTP project - the foundation of DJDNS, which democratizes DNS. I wasn’t sure what would happen, how big a splash in the pond it’d make. After all, the code was a bit fragile and rough around the edges, with no real practical stuff built on it yet. I’ve never managed a project of any kind in my life, short of a few college projects I did the majority of work in back in Folsom Lake.
It’s time to come back, see how things went, and where they’re going in the very immediate future.
The immediate rush
I remember the first two days. Danry25 and I posted the link all over reddit. I posted it on Google+… Twitter too, I think. Reddit started rate-limiting me, so I verified my email address, which actually does make the filter ease off your back. I’d made a half-hearted attempt at advertising this before, to basically no effect, so it was all in this time around. I remember watching the server access logs, getting record numbers, multiple hits per second. Nothing challenging, even for the most ancient and poorly-configured server, but a higher readership than this site has probably ever had before or since. No turning back now, things were in motion.
Previously, my Google Reader RSS feed for Github was dominated by news of Hawkthorne. For the next couple days, it was predominantly people forking and commenting and starring and following me and EJTP.
It wasn’t long before people started work.
Your code is broken
The first stuff to come in was not pull requests, of course. One of the consequences of being the sole user of your own project is that you have no idea what portability issues will rear their ugly heads the second anyone with a minutely different environment tries to install your software. One bug in particular, where PyCrypto changing an error description caused a test to fail, was reported twice. Another was the lack of python-dev being installed, which I hadn’t even thought of as a dependency, but is vital to setting up PyCrypto.
Other problems were less dependency-related. The whole codebase was a disaster in terms of Python 3 support, as I learned thanks to the soon-to-be-prolific programmer MoritzS. I never used Python 3. He used it every day. This ended up being one of the single biggest undertakings in the financed history of EJTP, and part of the reason why MoritzS has earned so much money off me! It paid off, though. EJTP now installs and tests perfectly in Python 3.2.
Paypal is terrible. It’s convenient, it’s ubiquitous, and it’s terrible.
One of the reasons I say this is the individual $5000 caps they have on paying and recieving money, which I had been edging closer and closer to for weeks. A lot of my money already goes right back out the door thanks to Kickstarter campaigns, monthly payments to WIKISPEED and the EFF, weekly payments to Kiva and Gittip… and basically all of that goes through my Paypal. Not to mention all the stuff I got people for Christmas. MoritzS’s fantastic efficiency and apparent need to claim and complete every single paid job was great in almost every sense, except that FreedomSponsors uses Paypal exclusively.
I was actually already waiting for PP to confirm my bank account, which takes away the caps. I was reluctant to, but the other alternative was to give them my online banking credentials including password. The fact they even ask for that is insane. The confirmation process involves depositing two small payments into your account by checking information, then withdrawing them. You look at your records in your online banking site, and use those numbers to confirm the account (as only the account holder will have access to that knowledge).
A few days into financed development, I got an email saying that the process was not successful, and Paypal had been unable to deposit any money.
This was bad for a ton of reasons. They’d burned around a week of my time on this failure, with no guarantee a second attempt would work. Was it just a typo on my part? Was my credit union just too obscure? I was closer to the limit than ever, and had no way to know if my second try would be processed in time.
I felt like a real sleeze relaying this information to MoritzS in Github comments, but he was really understanding. This took a real load off my mind, but thankfully, in the end I didn’t have to rely on anyone’s patience but my own ragged supply. Turns out it was a typo in my checking information. The second try worked and now I have a fully linked Paypal account with no caps. If I have money (and I do), I can pay people.
The damned Identity merge
The biggest thing I had to do was take the Identity system from DEJE and move it upstream to EJTP where it belongs. Unfortunately, this was a long and slow process, competing for my time with college classes and a full-time job (which is where I get the money to subsidize stuff like this). It inched along, holding up other development, and causing some painful merging. I was doing this in parallel with the entire Python 3 support arc, which encompassed 2 tickets and probably 5 branches. At the end, I had to take the two different directions and merge them together so that they worked, which is much easier said than done.
This was really indicative of most of my personal work on EJTP. I’ve barely had time to do stuff myself, which was a lot of my motivation for hiring people, but being a maintainer takes almost as much work (if not moreso). It’s a different type of work, and stuff gets done a lot faster, but it really popped my fantasy bubble about being able to worry less about my own duties. Now every bug report is on my shoulders, and every merge is in my hands. If I was attracted to power and had more time to wield it, I’d probably be a lot more enthusiastic about that, but as it is, I’m just a happy kind of tired.
Where we’re at right now
Despite all the stress and worry and pessism I felt, and expressed above, this has actually been fantastic!
EJTP has very rapidly become portable and much more reliable.
- Out of the five sponsored issues, only one is still open.
- Multiple serious bugs and crashes are now resolved.
- Code is already much cleaner and better-covered by tests.
- Development is now the official base branch of the repository.
- The live console was brought out of retirement, fixed up, and is now a shining example of how to load and use Identity data.
- There’s an official Identity cache format.
- Great Python 3 support.
- So far, I’ve paid $110 to MoritzS, and will pay Iuri de Silvio his $50 the second he marks that ejtp-console job as done (already merged into development).
- 21 issues opened during these 17 days of financed development - only 4 are still open.
- MoritzS has had 26 commits (1138 lines added, 490 subtracted) worth of his changes merged.
- Iuri de Silvio has had 33 commits (222 lines added, 76 subtracted) worth of his changes merged.
- Multiple other people currently have branches they are actively developing, but which are not merged yet.
- During that time period, my own contribution has been 21 commits (+725, -294).
- If my site had a hit counter, it would read at 44576, and 20370 of those hits would be the article 17 days ago that announced financed development.
I want to get these new features and increased stability into stable soon, so I’m announcing a merge window from now until February 1. If you can get your branch done before then, it gets into EJTP v0.9.2. If not, it has to wait until v0.9.3. The only one I think could really go either way is the compression feature.
For 0.9.3, we’re going to switch from doctest to the unittest framework. Thanks to the nature of DoctestAll, we can do this piece by piece, which means I can break that project up into multiple self-contained issues that people can work on in parallel. We’ll still use DTA to keep our in-docstring examples up to date, but will only use examples for human readability, and not as a way to verify the code. More like the other way round, with the code verifying the examples.
There will also be continued work on the provided script utilities, new jacks, and ejtpd. Probably the most exciting highlight of that is being able to create and edit identities in ejtp-console, though I’d also like to work on making the ejtpd system take advantage of identities, and conversion utilities between GPG data and identities. Finally, I expect to overhaul the router soon, to support arbitrary rules and filters based on callbacks, as this will pave the way for a future high-efficiency stateful stream system (like an SSL connection that can bridge over any arbitrary transport supported by EJTP). After the router overhaul, I expect most of the fires to be put out, and that the vast majority of the work can be sponsored without needing much intervention from me (besides administration).
In a less core-ejtp sense, I’m hoping to work with liamzebedee on peer exchange technology, and implement a vanilla Kademlia DHT component of DEJE. The dude’s got some interesting ideas, which he’s posted in /r/darknetplan, and he does his research.
If the pace keeps up like this, we’ll probably be able to maintain a schedule of a point release every month. I’ll bump it up to 1.0 when I think it’s ready.