Making a Bitcoin service: web hierarchical deterministic wallet

It looks like Bitcoin is developing really fast these days, both on the usage and on the technical side. There are a lot of usability issues, and ideas for bitcoin services that look great on paper but don’t exist yet. Many of those ideas will be created with time, some of them though it’s better to go ahead and make as soon as possible, to let people use and see the advantages and disadvantages.

One of the most interesting ideas I’ve read so far is the creation of the Hierarchical Deterministic Wallet, or Bitcoin Improvement Proposal (BIP) 32. It is trying to solve the problem that currently the standard Bitcoin clients need generate independent new Bitcoin addresses to the user, and store a piece of secret in a wallet file for every Bitcoin address a person has. If that file is gone and not backed up, the person will lose access to those coins permanently.

Others tried to fix this too, like Armory and Electrum, two clients that can generate a whole chain of addresses, though have their own limitations that I don’t go into (though at the moment I’m using Electrum for most my coins).

BIP32 on the other hand can generate infinite number of new address from a master secret, and all of that arranged in a hierarchy that one can create well separated accounts and addresses very easily. One example is creating a separate wallet for each of the branches of a store, or for different websites a person is working on.

One practical problem with BIP32 was that I couldn’t find any wallet management software for it. Electrum has that in the works (I think for the 2.0 version) but they are not there yet. Had to make one myself to try it out.

Design choices

I wanted to make something easy to use (as much as possible given the hairy details), secure, and powerful enough.

For ease of use, it’s a single web page that does all the work with Javascript and HTML; try to have as few moving pieces as possible; make sensible default choices, e.g. for the structure of the hierarchy.

For security the keys entered in the page are never transmitted over the network; the created transactions can be checked independently by a 3rd party (can decode it with Blockchain.info); the single page can be saved

For power it can use both public keys for querying balance, and private keys for actually preparing transactions; it generates addresses automatically; it has everything needed for transactions within one page, with very little external dependency; have access to advanced functions if needed.

Implementation

It took a few day, it it was too bad to get to a working prototype: it is hosted on http://webhdwallet.github.io/, but can be downloaded with all the code to makei it a stand-alone application.

I didn’t want to implement the BIP32 extended public/private keys generation because someone better than me already did it at BIP32.org, and it is also a good way to separate responsibilities. Two independent crooked website is much less likely than one, isn’t it?

The Bitcoin functions, including using the BIP32 keys, are delegated to the bitcoinjs library. Apparently there are a bunch of forks of the original one at various stage of advancement, and incompatible added features. I have chosen the fork one that looked the most active, by BitGo, to import into this project. So far so good, maybe will do some porting of features between the forks later.

The “standard” way of creating addresses from here is creating two chains: an external that the user is supposed to share with whoever wants to pay him or her, and an internal chain for change addresses (to eliminate address reuse as payments are sent). The site creates the bunch of these addresses starting from the 0th one (as computer programming so often start to count from 0).

All of these addresses are checked with the Blockchain.info JSON API whether they ever had any transactions. If they did, then check for the spendable coins, and generate some more addresses in the chain. This tries to reduce address reuse and ensure that all addresses used so far are checked. Of course, this is one of the weaknesses of BIP32 – one can never really be sure without a lot of computation or out-of-band communication whether all the addresses ever used with the key are accounted for.

If any spendable coins are found, then the user can create a new transaction. I didn’t put in too much effort into finding a good coin-selection algorithm, just start from the oldest one and add more to the input side of the transaction until there are enough to cover the desired outgoing amount. Apparently, though, that is a good way to do it, so fair enough.

If the extended public key is used, then the page only knows enough to create an unsigned transaction. This can be checked, and hopefully later I’ll be able to implement the feature of signing such transactions  (with the same page) when being offline for security. If the private key is present, then a proper signed, spendable transaction is made, and ready to be submitted via the Blockchain.info Broadcast Transaction tool.

The receiving addresses have QR generation too. The chain addresses not, because they shouldn’t be used directly – this is just some opinionated programming.

Usage

The incoming BIP32 keys are generated as it is described in the help section of my page: choose a hard enough passphrase, and generate a child key as custom m/i’ child path (this should really be standard, by the way). It just means take the master key (m), and create the ith child in “private derivation” mode (hence the prime). Can see the original BIP32 page for some of the details.

This leaves you with an extended public key (something starting with xpub…) and an extended private key (xpriv…). Should keep the passphrase from the previous step as well, but definitely these two keys.

When one of these keys is plugged into the page, it starts to generate the appropriate keys for the two wallets, and the balance shows up. When any balance is found, a new transaction can be created, and sent off to the network with Blockchain.info.

To prove that I made this work before at least, the extended public key to generate the listed donation address of 17NWCFWo8EvFp7vtkbRH6ec3DEdxZhrhrd is xpub69i6TTB6JU2mwcQ7pKeDG8aAMnc2AZ2UdpuphoNak4nT4UTWYhkSGqpDgbGjHGbxYVK8jNF4eXMRk1aeGweLxiCWWB5EjKm3k6YMKoWN5VT (receiving address chain index 1) – go ahead, try it. Also used the page to create a small transaction – sending the from receiving address chain index 0, which also used the change chain index 0 address. When that worked, that was a relief. :)

Future

There’s a lot to do about this project and related services to make it more usable and interesting, the Issue Tracker is bursting with ideas. Here are some with higher priorities:

• really make it offline usable (which would remove a lot of security concern, probably). Will need to think much more about the internal implementation then, how’s the most user friendly to create new transaction if you cannot go online to get data (what and how to import between online and offline)
• make a usage video
• generate addresses in the receive/change chain at arbitrary indexes (could be useful)
• add QR code reading to the input field, probably via jsqrcode.
• implement storage of retrieved data so it’s more user friendly when on non-public computer
• talk to the bitcoin network directly, either to get the input or to send the transaction. Some groundwork is laid down in a blogpost recently about using the raw Bitcoin protocol in Python.
• implement my own server/API for Bitcoin/Litecoin/Dogecoin that can be used with BIP32 wallets, and probably Blockchain.info compatible. Currently there are no good service for such altcoins (even if it would be pretty straightforward I think), and for the Testnet (so not risking real value to try things out)
• implement a Point-of-Sale app (I guess on Android), that uses an extended public key to generate receive addresses for incoming transactions (totally hold-up-safe, and crooked-employee-safe payment method)
• implement a WordPress plugin that uses extended public key to generate per-post donation addresses (for the donations themselves, as well as analytics-via-payment)

Well, at least the first step is done. All source up on Github. Would love to hear from anyone who used it, and what do you think could be improved upon.