Saturday, December 10, 2011

The truth about Android & iOS UI performance

Some fallacies related to Android vs. iOS UI performance made the rounds recently, inflicting undue damage to Android's reputation. It all started with a misinformed Google+ update written by former Android testing intern Andrew Munn.

Time reporter Matt Peckham echoed Andrew's misconceptions the following day, further spreading the FUD about Android. Toward the end of his article, Matt noted:

I’m not an Android or iOS software engineer, so all I can say in response to any of this is that, assuming Munn’s correctly articulated the way rendering takes places on Android and iOS devices, it makes sense (but then so does the idea that Lee Harvey Oswald had help, at least to some people).

Peckham makes no mention of trying to corroborate Munn's claims with a more experienced, knowledgeable engineer, like Romain or Dianne from the Android team, nor does he reference the corrections made by iOS experts in the comments on Munn's post. A more qualified engineer would support their theories with evidence like code, specifications, and performance test results, not Reddit and Hacker News comments as Munn did.

I don't claim to have all the answers, but I can tell you that implementing fluid interfaces on both iOS and Android is time consuming and difficult. The challenges are an order of magnitude more complex than Munn suggests. I haven't had an opportunity to try Ice Cream Sandwich yet, so I can't tell you firsthand how it compares to the iPhone. However, Jason Kincaid, quoted by Munn, described ICS as quite smooth and noted that both Android and iOS stutter occasionally.

Now that Android has pervasive hardware acceleration, Android has no fundamental technical disadvantage compared to iOS with regard to to implementing fluid UIs. Given comparable hardware, any UI that runs smoothly on iOS should be able to run just as smoothly on Android. That said, hardware acceleration isn't a magic switch that you can just turn on. Android developers need to modify their applications to take full advantage, just like iOS developers have to design their own applications with hardware acceleration in mind. This will take time.

To Munn's point, "UI thread" and "main thread" mean the same thing. Most user interface frameworks, including those employed by iOS and Android, have a notion of receiving input events and manipulating the user interface from a single thread. Users of those frameworks often use the terms "UI thread" and "main thread" interchangeably. Why impose this restriction? If you limit access to UI elements to a single thread, you simplify API usage and improve performance because you don't have to deal with locks and concurrency. Contrary to Munn's claims, Android does actually assign a higher scheduling priority to the UI thread than it does to background threads. I assume iOS does likewise.

Android and iOS are more alike than different in this respect. Either platform will hang and stop responding to user input if you monopolize the main thread with blocking operations (like file I/O) or slow drawing logic. This likely explains why Jason experiences jitters when visiting global search on iOS. On both platforms, programmers should perform all blocking operations on background threads and should never block the main thread.

As for the differences, iOS's Core Animation framework tweens animations in a background thread while Android does the same in the main thread. On iOS animations continue running, even when the application accidentally blocks the main thread. In addition, when scrolling, iOS's main thread goes into a "tracking" mode where it filters out certain types of events that might result in redrawing or other performance degradations. This has no affect on background operations. Nothing technically prevents Android from supporting these features, but their absence doesn't preclude smooth UIs on Android either. Hardware acceleration is far more important.

Does this mean we're going to start seeing iOS-quality user interfaces across the board on Android in the near future? No way. The reasons have little to nothing to do with bytecode or garbage collectors and everything to do with the community and tools. First, iOS app developers have far more experience taking advantage of hardware acceleration. They're experts in going out of the way to avoid software rendering. Hardware rendering requires a different mindset, and Android programmers will need time to catch up.

Second, Android programmers need to support both software and hardware rendering for awhile. This requires more code. Some Android devices support only software rendering while others, like the Xoom, actually require hardware rendering to achieve any semblance of smooth animation. Developing and maintaining smooth Android apps will require significantly more developer resources than accomplishing the same on iOS, at least until we can retire support for pre-Honeycomb devices. Programmers with limited resource will no doubt have to elide animations and settle for lowest common denominator solutions for awhile.

Third, from what I've seen, iOS developers have far better tools at their disposal. iOS developers can tweak and reload their applications in seconds, faster than you can reload a Ruby on Rails web page, while Android developers are lucky to do the same in tens of seconds. Being able to quickly iterate and tweak a UI is essential to achieving pixel-perfect, high frame rate animations. If Android programmers need to wait up to a minute after each tweak, they'll be at a significant disadvantage compared to their iOS counterparts. iOS provides amazing tools that overlay their UIs and help pinpoint and eliminate software rendering. In contrast, the UI performance and bugs in Android's emulator are so bad that developers resort to running on real devices only, even during development.

Android's training and legacy support issues will resolve themselves eventually. If I ran the Android team, I'd triple down on tool support. Android's emulator sounds great in theory, but it fails spectacularly in practice. Professional Android programmers simply can't and don't use it. Frankly, the emulation model is a lost cause. Android should replace it with an iOS-like simulator, one with tightly integrated profiling tools, capable of building and displaying apps in seconds. Ideally, the development build process would further cut down the turnaround time by eliding resource compilation, dex conversion, etc.

In the meantime, Android developers can achieve buttery user interfaces on Honeycomb and ICS devices, just like their iOS equivalents, by eliminating blocking operations from the main thread, minimizing time spent drawing in software, and allocating objects judiciously. Android programmers just have to work a little longer and harder to get there. Given our fierce dedication to a flawless user experience, my team plans to invest the necessary resources and accomplish just that. If you think you're up to the challenge, please apply.

For additional reading, Dianne has done a wonderful job explaining why the Android team made the choices they did and how those choices enabled some unique advantages over iOS.

Tuesday, November 15, 2011

Coding Challenge: The Luhny Bin

Note: This is cross posted from Square's engineering blog. Please submit comments over there.

“To err is human; to forgive, divine.” -Alexander Pope

Mistakes happen. At Square, we accept that human error is inevitable. We anticipate potential slip-ups and implement safety measures to mitigate—and oftentimes completely eliminate—any repercussions.

For example, Square’s Luhn filter monitors logs and masks anything that looks like a credit card number. If a number like “4111 1111 1111 1111” were accidentally logged as part of an error message, our filter would replace it with “XXXX XXXX XXXX XXXX” and page an on call engineer.

The Luhn filter looks for sequences of digits that pass the Luhn check, a simple checksum algorithm invented by Hans Peter Luhn in 1954. All valid credit card numbers pass the Luhn check, thereby enabling computer programs, like our log filter, to distinguish credit card numbers from random digit sequences.

The Luhn check works like this:

  1. Starting from the rightmost digit and working left, double every second digit.
  2. If a product has two digits, treat the digits independently.
  3. Sum each individual digit, including the non-doubled digits.
  4. Divide the result by 10.
  5. If the remainder is 0, the number passed the Luhn check.

For example, “5678” passes the Luhn check:

  1. Double every other digit: 10, 6, 14, 8
  2. Sum the individual digits: (1 + 0) + 6 + (1 + 4) + 8 = 20
  3. Divide the result by 10: 20 mod 10 = 0 Pass

“6789” does not:

  1. Double every other digit: 12, 7, 16, 9
  2. Sum the individual digits: (1 + 2) + 7 + (1 + 6) + 9 = 26
  3. Divide the result by 10: 26 mod 10 != 0 Fail

Now for the challenge…

Write a command line program that reads ASCII text from standard input, masks sequences of digits that look like credit card numbers, and writes the filtered text to standard output. For the purposes of this challenge, a credit card number:

  • Consists of digits, spaces (' ') and hyphens ('-').
  • Has between 14 and 16 digits, inclusive.
  • Passes the Luhn check.

If a sequence of digits looks like a credit card number, replace each digit with an 'X'. Any characters, including digits, may flank a credit card number. Beware. Potential credit card numbers can overlap. A valid 16-digit number can even contain a valid 14 or 15-digit number. Your program must mask every digit.

I already wrote a test suite, so you can jump straight to the fun part: writing the algorithm. To participate:

  1. Fork the Luhny Bin GitHub repo.
  2. Modify mask.sh to call your program.
  3. Test your program by executing run.sh.
  4. Once run.sh passes, post a link to your solution in the comments below.

Windows users should use Cygwin to run the tests. Please make it easy for others to check out and run your solution.

The first time you execute run.sh, you’ll see a test failure:

$ ./run.sh 
Running tests against mask.sh...

.X

Test #2 of 20 failed:
  Description:     valid 14-digit #
  Input:           56613959932537\n
  Expected result: XXXXXXXXXXXXXX\n
  Actual result:   56613959932537\n

Modify mask.sh and make the tests pass. Line feeds delineate the test cases. If you pass a number on the command line, run.sh will repeat the test suite the specified number of times; this is useful for performance comparisons. The tests aren’t set in stone—if you have an idea for improving the test suite, please submit a pull request.

This isn’t a contest, but an innovative solution could score you interviews at Square. I’m primarily interested to see how different programming languages stack up with regard to readability and performance.

Once we have enough interesting submissions, I’ll summarize the results in a followup blog post and open source our own Java-based implementation. In the mean time, if you enjoy working with talented people on challenging problems like this, email your résumé to luhnybin@squareup.com.

Good luck!

Tuesday, November 16, 2010

Java SE 7 & 8

Sometimes doing the right thing can be extremely difficult. In spite of and because of my commitment to Java, I had to decline an invitation to join the Java SE 7 and 8 expert groups. Talk about a tough decision!

Why did I decline? I'll contribute to open source projects and open standards, standards I can independently implement and license as open source. Java SE, with its anti-competitive licensing restrictions, is not an open standard. It doesn't belong in the thus far open JCP, and I can't support such a charade. I stand by Apache, and I hope Eclipse comes around.

I'm not a lawyer, but I can't help but wonder whether Oracle even has a right to apply additional licensing restrictions to the intellectual property in Java SE. Oracle didn't create Java SE on their own. Look at the list of JSRs pulled into Java SE 7 and SE 8 (see the Contributions sections). Look at the experts on those JSRs. Most of them aren't from Oracle. For example, JSR 310: Date and Time API comes from our own Stephen Colebourne.

The JSPA section 4.D protects experts in these situations, enabling them to withdraw their IP contributions if a spec lead, Oracle in the case of Java SE, changes the licensing terms:

Withdrawal of Contributions due to Change in Announced License Terms. If the Spec Lead for an Expert Group in which You are participating makes significant changes to the terms and conditions of any license granted pursuant to Sections 5.B or 5.F below after those terms and conditions are first disclosed pursuant to the Process, then You may, upon giving notice to the Specification Lead and the PMO, withdraw any copyright or patent licenses concerning Your Contributions granted pursuant to Section 4.A.I or 4.A.II above (but not patent licenses granted by You pursuant to Section 6 below).

It's not clear to me exactly how section 4.D applies to umbrella JSRs like Java SE, but I'd say that changes preventing independent open source implementations qualify as a "significant."

Monday, November 01, 2010

GOTJCPECV (Get Out The JCP EC Vote)

Today is the last day to vote in the JCP EC election. If you're a JCP member, you received voting instructions in an email from pmo@jcp.org with the subject line JCP Elections 2010 end November 1! This email contains the link and your password.

Note: The password is different from your jcp.org password.

If you voted already, thank you. Please help me spread the word through blogs, tweets, Facebook, etc.

Thursday, October 21, 2010

Square Snare

My dad makes and sells keychains for Square readers. He attaches a keyring to a 3.5mm jack. Make your own or order one pre-made for $5:

Wednesday, October 20, 2010

Long Live Java

The JCP EC elections opened today, and I'm running.

After Doug Lea's resignation and subsequent replacement with Hologic, Inc., maintaining individual representation is more important than ever to Java's future. Individuals stand to account for only 13% of the SE/EE EC. Please vote for me and hold the ground for non-corporate interests.

Voting

If you're a JCP member, you received voting instructions in an email from pmo@jcp.org with the subject line "JCP Elections 2010: vote today." The email contains a link and a password. If you have any problems, email admin@jcp.org or call 1-866-543-8750 (US or Canada) or 1-202-207-0529 (International).

Why vote for me?

You're no doubt aware that a bargaining impasse between Apache and Sun/Oracle has virtually halted progress within the JCP over the past five years. Openness of Java SE is critical in the fight against its real competition: .NET, V8, Erlang, Flash, et al. I'll do everything I can to ensure an open future for Java, but we can't let one JSR grind the JCP to a halt. Life must go on.

Just last year, in the midst of this kerfuffle, I led the fastest-executing, most open JSR in the history of the JCP:

If you blinked, you might have missed it. JSR 330, Dependency Injection for Java, completed the shortest development cycle of any Java Specification Request on record within the Java Community Process (JCP) program, a feat that looked only theoretically imaginable on paper until Bob Lee proved it could be done. As co-Spec Lead of JSR 330, Bob wasn't focusing specifically on agility when he pushed the envelope. "While I'm glad JSR 330 executed quickly, I'm even prouder of our specification's quality. I think we raised the bar," he says.

How is it possible to achieve both speed and superiority? Doesn't something have to give? In fact, Bob gave a lot to make this happen, and other Spec Leads can learn a lot by retracing his steps. He attributes the short execution time and enhanced quality to a number of factors, including a crack team of Experts, an aggressive schedule that hacked the normal process, transparency, limited scope, and a willingness to collaborate, compromise, question assumptions, and think outside the box.

If you elect me...

I'll continue this record of innovation and pave the way for more JSRs like 330. I'll introduce an abbreviated process for Open JSRs. JSRs that meet our criteria for openness will be able to follow an easier, faster, simpler specification process. This process will motivate more open JSRs. It will free spec leads up to focus more on the technology and less on the process. It will attract more experts to the JCP and ensure a bright future for Java.

Thank you,
Bob Lee

Friday, February 05, 2010

Android: Trusting SSL certificates

We use a self-signed SSL certificate for the test version of our backend web service. Since our certificate isn't signed by a CA that Android trusts by default, we need to add our server's public certificate to our Android app's trusted store.

These same instructions apply to trusting a custom CA, except you'd get the public certificate directly from the CA instead of from a server.

Required tools:

1. Grab the public certificate from the server you want to trust. Replace ${MY_SERVER} with your server's address.

echo | openssl s_client -connect ${MY_SERVER}:443 2>&1 | \
 sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' > mycert.pem

For example, here's the PEM-encoded public certificate from google.com:

-----BEGIN CERTIFICATE-----
MIIDITCCAoqgAwIBAgIQL9+89q6RUm0PmqPfQDQ+mjANBgkqhkiG9w0BAQUFADBM
MQswCQYDVQQGEwJaQTElMCMGA1UEChMcVGhhd3RlIENvbnN1bHRpbmcgKFB0eSkg
THRkLjEWMBQGA1UEAxMNVGhhd3RlIFNHQyBDQTAeFw0wOTEyMTgwMDAwMDBaFw0x
MTEyMTgyMzU5NTlaMGgxCzAJBgNVBAYTAlVTMRMwEQYDVQQIEwpDYWxpZm9ybmlh
MRYwFAYDVQQHFA1Nb3VudGFpbiBWaWV3MRMwEQYDVQQKFApHb29nbGUgSW5jMRcw
FQYDVQQDFA53d3cuZ29vZ2xlLmNvbTCBnzANBgkqhkiG9w0BAQEFAAOBjQAwgYkC
gYEA6PmGD5D6htffvXImttdEAoN4c9kCKO+IRTn7EOh8rqk41XXGOOsKFQebg+jN
gtXj9xVoRaELGYW84u+E593y17iYwqG7tcFR39SDAqc9BkJb4SLD3muFXxzW2k6L
05vuuWciKh0R73mkszeK9P4Y/bz5RiNQl/Os/CRGK1w7t0UCAwEAAaOB5zCB5DAM
BgNVHRMBAf8EAjAAMDYGA1UdHwQvMC0wK6ApoCeGJWh0dHA6Ly9jcmwudGhhd3Rl
LmNvbS9UaGF3dGVTR0NDQS5jcmwwKAYDVR0lBCEwHwYIKwYBBQUHAwEGCCsGAQUF
BwMCBglghkgBhvhCBAEwcgYIKwYBBQUHAQEEZjBkMCIGCCsGAQUFBzABhhZodHRw
Oi8vb2NzcC50aGF3dGUuY29tMD4GCCsGAQUFBzAChjJodHRwOi8vd3d3LnRoYXd0
ZS5jb20vcmVwb3NpdG9yeS9UaGF3dGVfU0dDX0NBLmNydDANBgkqhkiG9w0BAQUF
AAOBgQCfQ89bxFApsb/isJr/aiEdLRLDLE5a+RLizrmCUi3nHX4adpaQedEkUjh5
u2ONgJd8IyAPkU0Wueru9G2Jysa9zCRo1kNbzipYvzwY4OA8Ys+WAi0oR1A04Se6
z5nRUP8pJcA2NhUzUnC+MY+f6H/nEQyNv4SgQhqAibAxWEEHXw==
-----END CERTIFICATE-----

2. Android has built-in support for the Bouncy Castle keystore format (BKS). Put Bouncy Castle's jar in your classpath, and create a keystore containing only your trusted key.

export CLASSPATH=bcprov-jdk16-145.jar
CERTSTORE=res/raw/mystore.bks
if [ -a $CERTSTORE ]; then
    rm $CERTSTORE || exit 1
fi
keytool \
      -import \
      -v \
      -trustcacerts \
      -alias 0 \
      -file <(openssl x509 -in mycert.pem) \
      -keystore $CERTSTORE \
      -storetype BKS \
      -provider org.bouncycastle.jce.provider.BouncyCastleProvider \
      -providerpath /usr/share/java/bcprov.jar \
      -storepass ez24get

3. Create a custom Apache HttpClient that uses your custom store for HTTPS connections.

import android.content.Context;
import org.apache.http.conn.ClientConnectionManager;
import org.apache.http.conn.scheme.PlainSocketFactory;
import org.apache.http.conn.scheme.Scheme;
import org.apache.http.conn.scheme.SchemeRegistry;
import org.apache.http.conn.ssl.SSLSocketFactory;
import org.apache.http.impl.client.DefaultHttpClient;
import org.apache.http.impl.conn.SingleClientConnManager;

import java.io.InputStream;
import java.security.KeyStore;

public class MyHttpClient extends DefaultHttpClient {

  final Context context;

  public MyHttpClient(Context context) {
    this.context = context;
  }

  @Override protected ClientConnectionManager createClientConnectionManager() {
    SchemeRegistry registry = new SchemeRegistry();
    registry.register(
        new Scheme("http", PlainSocketFactory.getSocketFactory(), 80));
    registry.register(new Scheme("https", newSslSocketFactory(), 443));
    return new SingleClientConnManager(getParams(), registry);
  }

  private SSLSocketFactory newSslSocketFactory() {
    try {
      KeyStore trusted = KeyStore.getInstance("BKS");
      InputStream in = context.getResources().openRawResource(R.raw.mystore);
      try {
        trusted.load(in, "ez24get".toCharArray());
      } finally {
        in.close();
      }
      return new SSLSocketFactory(trusted);
    } catch (Exception e) {
      throw new AssertionError(e);
    }
  }
}

That's it! If you think this kind of stuff is fun, Square is hiring.