First off, we would like to give a massive thank you to anyone and everyone who reached out to us in the past week or with messages of support. I tried really hard to respond to every message but there were simply too many. Know that we read each and every one of them and if we didn't respond to you and you want a response, send the message again. It's been pretty chaotic but now that the dust has settled, we're in pretty high spirits. Now, onto the wall o' text!

Better "URI" support

You can use query strings on the Send Transaction page in order to pre-fill in information for a user. We have always supported this but it's been updated to support sendMode and tokenSymbol:

to=[address]
- default is empty
value=[number]
- default is empty
sendMode=0 | 1 | 2 | 4
- 0: ETH (Standard Transaction) 1: Only ETH 2: Only ETC 3: We have no freaking idea 4: Tokens
- default is 0
tokenSymbol = ARC | ICN | MKR | ....
- default is whatever the first token in the list is if sendMode is set to 4
- BeerCoin and Unicorn are those words, sans emoji
gasLimit=[number] OR gas=[number]
- default is 21000 and then estimated once a value is inserted. Get in the habit of using this though as our embedded send page will not estimateGas for users.
data=[hex data]
- default is empty

Notes:

Nothing is case sensitive.
All fields are optional.
Token Symbol takes priority over sendMode. So no need to include sendMode if you have a tokenSymbol
gasLimit takes priority over gas if both are included
Data can only be included in a standard transaction (sendMode=0). If another sendMode is specified, a warning displays for the user.

Example Links:

Misc

Pressing the enter key will now unlock your wallet so you don't have to click the button. This has already saved me about 2000 button clicks. It's pretty much the best thing ever.

The Hardfork

Our backend is already set up for the hard fork so we're ready to roll no matter what. The only thing we are still contemplating is how to handle EIP-155 (replay safe TX / new TX signing method). We are ready to switch transaction signing to match the EIP-155 spec. However, we are trying to decide what to do with ETC. Here are the options:

Plan A:

Standard Tx will follow EIP-155
Remove Send Only ETH
Remove Send Only ETC from site (until they implement their version of EIP-155?)
Tell people to use an old version of the repo (https://github.com/kvhnuke/etherwallet/releases/tag/v0.3.2.6) if they want to Send Only ETC.

Plan B:

Standard Tx will follow EIP-155
Remove Send Only ETH
Send Only ETC will use old syntax to send via replay safe split contract

Plan C:

Leave as is and don't use new transaction EIP-155 signing (until we have more information to make a decision.)

Plan A is preferred simply because the Send Transaction controller is already a mess and keeping the old send transaction code makes the entire thing more messy. The problem with Plan A is I really don't feel like answering a bunch of emails from people who want to use the site to Send Only ETC.

Plan C is only appealing to allow us to know if ETC is going to get on the EIP-155 train in a timely manner. If they aren't, we would go with plan A. The only thing worse than having messy code is having messy code indefinitely.

Weigh in on which plan you hate the most and tell us what to do.

Golem Crowdfund Analysis

As some of you are aware, we could not handle the traffic during the Golem crowdfund ~10 days ago. Up until the Golem sale, our server has occasionally crashed. The instances were few and far between and never happened at such inopportune moments. The first time we recall is when the DAO Token Creation started. Server crashed, we restarted, and everything was dandy in < 5 minutes. When we were on the flight home from Devcon2, it crashed again. This time it was due to the network attacks. We restarted, everything was fine.

Since we crashed during the DAO, our setup has changed completely. Everything was re-written in node.js, our servers have grown, and we have switched from geth to Parity.

When we started receiving requests for supporting Golem and learned about the data value necessary to successfully participate in the crowdfund, we made a few changes in preparation. We increased size of our nodes, we added functionality to automatically add data when the Golem address was entered, we increased our gas price to 61, rather than the 21 we were using before, and we troubleshooted some issues regarding estimating gas when there is data. If you notice, some of these items are frontend items that have no affect on our node. That's because I was more worried about transactions not being mined or people forgetting to include the data. I'm not a server person. I'm a user experience person.

We expected there to be a increase in traffic, but not by this much. We went to sleep at 4:45am and my husband woke me up at 6:30am when he couldn't get a test transaction to send via his Ledger (fyi, if a TX has data you need to turn on contract data on your Ledger). When the token sale started and the balances started loading slowly, I knew there was a issue. But with contract at 50% of it's cap and the server not returning errors or giving a 503, there was nothing we could do. Restarting would have done nothing besides ensure than no transactions could go through.

The last 6 or so token sales have gone off without a hitch. We actually slept through most of them and I we received no reports upon waking of issues. The only reason we knew there was a token sale is there was a load of emails and tweets asking for adding that token as a default token. Golem was different, by a significant margin.

There are a few main factors that we believed contributed to the latency of our server:

We had combined "Send Ether" and "Send Tokens". This means that anyone wanting to send Ether also sent getBalance requests for tokens as well.
getTransactionData is 3 calls - it returns the nonce, balance, and gas price so you can make a transaction. This call is one of the more expensive calls and occurs when you are actually generating a transaction.
There were an insane amount of people attempting to send Ether and therefore calling getTransactionData and getBalance of ETH and Tokens in their account...all simultaneously.

We should take this time to point out that our servers didn't crash like we've seen before. Parity was fine and synced the entire time. The server didn't drop any requests. The problem was the time it took to deliver said requests. That is why some users reported pressing a button and it generating the transaction 12 seconds later. This is known as latency or lag in the backend world and happens when there are a buttload of requests in the queue.

We should also take the time to point out that our frontend (what you see) is completely separate from our backend servers that run Parity. The frontend is served via github / cloudflare and never goes down for that reason. So, unlike a more traditional site, reducing the size or format of the frontend plays no role in how our servers handle requests.

The Data

http://i.imgur.com/HQHEBSK.jpg

For the 7am-8am period, we had 708,490 requests. Meaning, there were 708k requests for things like getBalance or getTransactionData.
700k/hour gives us an average of 200 requests per second. However, the crowdfund lasted less than 30 minutes, means it is likely that the number of requests per second was even higher during the initial 30 minutes than the latter 30 minutes.
There were 1264 unique visitors. Unique visitors doesn't count the dude who has 3 or 4 tabs open or refreshes upwards of 10 times during that hour, nor does it count anyone not on the https://myetherwallet.com version of the site (e.g. local users, chrome extension users).
It took ~700 transactions to reach the cap. So <700 people got to participate successfully.
It takes 0.065s to respond to a single getTransactionData call.
It takes ~6.57s to process 950 simultaneous getTransactionData
We can handle 900-950 simultaneous requests. Once it goes above that, the server becomes unresponsive.

At the end of the day, that is an enormous amount of people and requests and far higher than we've seen before. I fully expected to run tests and discover that we were woefully unprepared and our servers fall embarrassingly short of what they should be. They don't. Our performance is not subpar, especially considering we have spent far less relative time perfecting our servers as we have other items on MyEtherWallet. That's not to say we can't be better - I just blame myself a tiny bit less after seeing the stats.

Changes to MyEtherWallet to prevent this from happening in the future

Embeddable send page specifically for token sales / crowdfunds will be implemented. This will be similar to http://www.myetherwallet.com/embedded.html but for the send page. It will only accept URLs with query strings for sendMode or tokenSymbol and only load the ETH and/or selected token balance. It will not estimate gas. See the "Better URI support" section for details.
Lightweight mode will be turned on automatically during high-traffic times. This will remove all balances from loading upon wallet unlock, and a user will need to click in order to view their ETH or Token balance.
Changes to give user feedback on the status of their request, even when the server is under extreme load. Giving immediate feedback rather than giving users uninformative errors 30s after a request is sent. The fact this wasn't implemented long ago is fucking bonkers and we're sorry.
Further upgrades to servers and implementation of a proper load balancer to support at least 2x what we current support.
Setting up a "premium" page will act as another "load balancer". Transactions sent via this page will be sent via completely separate backend and only transactions with a fee paid to MyEtherWallet will be accepted. Since 99% of users don't pay, if you do you'll be part of the 1%, ensuring your transaction goes through. ;)
Set up / write out an easier way to run MyEtherWallet connected to custom / local node.
Explore decentralized server option where MyEtherWallet would connect to a list of trusted available nodes.

Recommendations

Our first set of recommendations are for people, like Golem, who are holding crowdfunds / token sales.

Do anything you can to prevent a mad rush for tokens in a limited amount of time. If MyEtherWallet had operated flawlessly, there is a good chance the sale would have been over in <5 minutes. ~700 people got in on the Golem sale via all methods of sending (mist, geth, MyEtherWallet.....). MyEtherWallet had 1200 unique visitors. This is not ideal.
Write up clean instructions on all the different ways a user can participate in your token sale. Don't recommend any one in particular.
Recommend your users have a contingency plan in place. Maybe MyEtherWallet goes down. Maybe Mist won't sync. Maybe your computer explodes.
Recommend they test far before the token sale and make sure everything goes as expected.
Recommend your users sign a transaction before the token sale begins and simply broadcast it during the token sale. This also allows the user to double-check their values in a calm environment and eliminates fat finger problems that can occur when the adrenaline is pumping.

If you are a user wanting to prepare for a sale you know will sell out fast....

Have a contingency plan in place. Don't rely on any one method. Maybe MyEtherWallet goes down. Maybe Mist won't sync. Maybe your computer explodes.
Test far before the token sale and make sure everything goes as expected.
Sign a transaction before the token sale begins and simply broadcast it during the token sale.
Attempt to run a full node. Stop saying that you're the average user who doesn't need to run a full node. You aren't. You're part of Ethereum in the early days. Your feedback on all clients is valuable and necessary as the ecosystem grows. Get yourself set up and help debug the issues you had while getting set up. Parity and geth have vastly improved in both stability and sync times. Start from scratch after this hard fork and learn something new.

tl;dr: I don't blame you. I don't expect anyone to read this whole thing but didn't feel like making multiple posts. Read the section that pertains to what you care about...or comment some blasphemy without reading any of it. 😎