Recent Honk Development EntriesGet RSS Feed

We are in the middle of another round of usability testing and after having many conversations about it this week, I wanted to share some of our learnings and ideas:

Ideas for Recruiting Usability Testers:

1) Look Around: We’ve had a lot of luck getting some amazing feedback from people we work near everyday. It’s a great way to introduce other people in your office building to your website and at the same time get some valuable feedback.

2) Ask the Pros: We’ve met many rock-star product and UI people at partner companies, customer companies, or even competitors. Surprisingly, many have responded when we humbly asked for their professional opinion of our site or a new design concept.

3) Be mobile: Sometimes, it is a pain for people to come into the office, so when we are flexible and offer to meet them at a coffee shop, restaurant, office, or class room….we have good luck and sometimes they will do it for free.

4) Be heard: Don’t be afraid to post on both physical (Panera Bread, Retirement facilities, Borders, etc,) and online bulletin boards. We’ve had the most success posting to Craigslist and offering a gift card to a fun store.

5) Be smart about your rewards: If you spend a little extra planning time, you can buy discounted gift cards or gift certificates in bulk.

6) Screening: If you are looking for a certain type of user, try using Survey Monkey or Zoomerang to get more information about potential testers before you invite them in.

Usability Testing Environment

1) Location: It’s best to be somewhere that the usability tester will not get distracted, but also helps if you are not in a completely quiet or lab-like environment, where the person feels uncomfortable or under a micoscope.

2) Give up all control: Let the user type URLs in and search on their own. You learn a lot about how people will actually enter your site, misspell words, use the mouse vs. shortcut buttons, etc.

3) Photographic Memory: If the user agrees, I would recommend using a screen and audio recorder…after sitting with about 5 users, things will start to blend together for you and your test session notes can be difficult to decipher, because the user is often moving faster than you can type or write (and you always want to be watching them).

4) Be clear about the time line: If a user thinks that the faster they blow through tasks, the earlier they get their reward, your results will not be as authentic. Tell them they have 5 minutes to complete a form, or that for the next 10 minutes you will be getting feedback about feature X, etc. It is also important to let them know they will be finished at a certain time, so they do not get impatient or wonder when the session will end. In my experience, 45 minutes is a great time frame for the testing and I usually plan for 5 minute prep and info gathering with a 5 minute end of test survey and a de-brief. Total time: 55-60 minutes.

Conducting the Usability Test

1) Scenarios: Develop about 10 fairly broad scenarios (ex. “You then search for a car based on your hobbies and important vehicle features.”) and a list of follow-up questions for each scenario after you have observed the user (ex. “What did you think about he list of hobbies?”, “Were the suggested vehicles reflective of what you searched for?”, etc.). This way, you can get good feedback even for people who are not very vocal. Not every person will need the follow-up questions, but they help when you are facilitating the usability test.

2) Set them at ease: Make sure the testers know that there is no wrong answer. “This is not a test. The only thing we ask is that you would use the site just as you would normally use any other site. Be honest about what you like and don’t like. You will not hurt our feelings. Learning from you is helpful and will only make our site better and our company more successful.”

3) Starting off: Resist the temptation to talk about the purpose of your company or website. You don’t want to guide the tester in any way. If you need to talk, just ask questions or talk about the process/time line.

4) Don’t help unless you have to: Sometimes it is hard to see somebody looking right at the big red button in the middle of the screen and then click on the small blue link on the top right of the page. You want to point to the screen and scream “it’s RIGHT THERE”. Don’t. Let them struggle, let them click on the wrong things and try to find their way back. When they are done, revisit and ask them why they looked at the link instead of the button. You will learn the most from these “alternate paths”.

5) Keep focus: If a tester gets distracted by something on your website or wants to talk about their new puppy, take note and gently guide them back to the task at hand.

6) Don’t be emotional: At a start-up, often times the people who work on the product are the same people hearing about what people hate. Remember that you are building the product for the user and not your personal taste. Be thankful for their honesty.

7) Don’t lead: Be careful when introducing concepts or designs to the user. For example, instead of asking “Do you like this design or the new design better?”, stay neutral and don’t talk about which is newer: “Tell me what you like and don’t like about each design. Which do you prefer and why?”

Take good notes: The data you gather during usability testing is valuable and you may need to refer back to it at times. Keep good records.

Providing a rich user experience calls for good client side software. Javascript is a powerful language to provide the needed functionality. Like any other tool it has it’s strengths and weaknesses. Successful design in Javascript favors composition, loose coupling, and separation of concerns more so than in most other languages. To see why, let’s look at some of the properties of Javascript.

Javascript is a prototype language with loose and dynamic typing. It is a flexible language. You have anonymous functions. You can alter objects at any time. You can aggregate functions from many sources. It provides closures and dynamic invocation. Also, each web page can be treated as a separate “application”, so you often do not need to have too many interacting pieces.

Javascript can also be a difficult language to use. Javascript does not give you a dependable stack trace. The debuggers often leave much to be desired. The weak type system makes it difficult to discern objects from each other. Inheritance is difficult.

These characteristics make it beneficial to build via composition. It makes sense to have loosely coupled objects. It also makes sense to unit test your Javascript software. It makes sense to craft the code to be scannable.

That being said, I really enjoy developing in Javascript because it encourages (or forces) me to better design my code. Working with a good testing framework leads to a quick, red/green/refactor cycle.

The Maintenance Costs of Complex Objects

Unfortunately, it is difficult to build an example because complexity in Javascript requires complex code. So I’m not really going to show code examples, but talk at a higher level.

Since Javascript does not provide good stack traces it becomes very time consuming to debug complex functions. To combat this problem, you should take advantage of Javascript’s flexibility and make loosely coupled functions that are composable. Good abstractions are important.

A good example of this is jQuery.

$(“.person”).click(function() {$(this).toggleClass(“active”);});

When an element with the ‘person’ class is clicked, the ‘active’ is toggled. Now imagine doing this with the old imperative style of Javascript. I could write the example, but it hurts to write and read, so I will spare you of it.

The more you can express in one line or one thought, the better.

Now these concepts are do not need to be constrained to DOM manipulation and hooking up events. You can add client/server concepts to the mix. Need to handle the case where the user can click on filters, each of which creates an Ajax request to update the search results? Don’t inline the code, create a function to do this. Give it a memorable name, like lastAjaxWins (not that that is the best name ever, but it is a memorable phrase).

The idea is to have consistent levels of abstraction. It may be expedient to have a function that has inconsistent levels of abstraction, but Javascript will not be very forgiving when you need to maintain and debug your code.

Fortunately, if you unit test and have loosely coupled objects and functions, your code will be easier to think about and your will not need to debug as much.

Heuristics to Measuring Complexity

Javascript makes complexity especially bad. When developing Javascript software, it helps to have some heuristics to keep the complexity down.

Talk About It

Whenever I complete a piece of functionality, I try to recreate the mental model of where this piece fits in on the web page. It should be clear, distinct, and easy to talk about. Talk about it with somebody. Is it easy to summarize?

If not, it is probably too complex. Break it apart. Create functions and objects which you can name. Compose these names to describe dependent objects.

Use Your Test Framework

A good javascript test framework (a couple of examples are Screw Unit and Jasmine) allows for nested ‘describe’ blocks and ‘it’ blocks (nested SUT’s, contexts, and test cases). When writing the name of the ‘describe’ block and ‘it’ block, try to capture the system under test, the situation of the test, and what happens. Be descriptive and precise. Don’t be vague. Being vague is cheating.

This is important because the length and complexity of your statements are a good indication of the complexity of the functionality you are testing.

Are the tests very painful to write? Are there repitive paterns? If yes, then that may be an indication that you need to extract something.

Refer to your Mental Model

Think about the client/server interaction. Whenever you add functionality, think about how it fits into the overall picture. Unless you are doing really heavy client side development, it should be easy to think about how that object and/or function fits into the picture. If you are doing heavy client side development, use good abstractions. Talk in the language of your abstractions. Let the concepts permeate through your consciousness. Having an intuitive grasp on your past, present, and future problems and solutions.

Following our Facebook App – The Good, Bad, and Ugly

From time to time, I’ll write about a specific topic from the start up world of Honk.  The goal is to share war stories and thoughts from the battlefield.  While the start up world is tough, it’s exhilarating – both the ups and downs. I’ll share whatever is on my mind – honestly and truthful.

We launched our Facebook application in December.  As with anything new, there’s so much learning that goes on.  While we all hope for some super out of the gate success, we are all realistic on how the game truly works – try, fail, try, fail, try, learn a little, try…. you get it. Our mission is to figure out how to unlock Facebook as a good communication tool for people shopping for a car. We know our long term plan is sound, but it’s excruciatingly frustrating going through the germination period.

Our initial app is called Cars I’ve Owned (CIO) which enables users to share stories and memories of the various cars in their virtual lifetime of garages. The goal of the app is to enable users to see which cars their friends own, have driven, or even dream of owning. The overarching value is clear – once we can establish who owns what, we can enable users to see which cars friends own so they can ask question about them (i./e., “How do you like your Civic?  I’m thinking about getting one”).  The value is there and we’re out to unlock it.

What have we seen to date?

We’ve had roughly 30K users who have added the app in the first couple of months.  The good news is that a good percentage of these folks end up coming to our destination site honk.com and are valuable users (although 15K is a relatively small sample size).  More good news is that people seem to be filling out their cars.  The bad news is that we haven’t seen a lot of viral-ity to the app yet. Users are checking out friends cars, but they’re not building their own set of cars.  We need to do a better job of getting in front of those users to add their own cars. We’ll work on that one for sure.

We added a “Dream Car” feature on the app, but it hasn’t yet fully caught on – at least in a viral nature.  While I know the key will be to get into the stream, we find a lot of people who like to post their cars, aren’t really inspired to publish them. We have to crack this code and will continue to test!  I wonder if people really do want to share their cars (I would hope so), especially a dream car.  There’s no harm in it – but the benefit could be great.  I’m just glad Facebook doesn’t ask for more details about users on their profile… it gives us all a chance to fill a void.

I have to believe that Facebook is going to be a great, structured communication platform.  People spend way too much time playing games and scanning photos… the evolution has to be meaningful Q&A and purpose-built tools that solves life’s problems by using friend networks.  It’s only a matter of time… and FB is the only game in town who can truly enable this.

More thoughts on this to come….

Here at Honk, we have been transitioning away from pair programming. I do not think that this necessarily reflects badly on pair programming. We still do some pair programming. However, our current circumstances steered us away from pair programming being our primary development practice.

Pair programming is a very powerful way to transfer and increase the skill of your developers. During my time at Pivotal Labs (and on the first few months at Honk), I have made leaps in my design skill, development speed, and bug reduction. I cannot think of a better way to transfer skill. It is easy to learn from someone when you are in the act of solving a problem.

Accelerated learning is something that cannot be overstated, and thus makes pair programming a very sustainable process in the right circumstances. You can also concentrate two minds on a single problem. Compared with soloing, you have half the tracks of development with more concentrated thought on each track of development.

However, we currently have a situation where we have few developers and many problems to solve. We simply would not have enough tracks of development, given the size of our current team. There are two solutions, hire talent or make due with your existing talent. Unfortunately, finding the right fit is very difficult in an industry where there is not enough supply. So for now, we have to make due and try to get more done with the same number of developers.

Going back to primarily solo programming has been an interesting experience. I certainly feel less time pressure, and more empowered to focus on making better quality software. I think this has to do with my propensities, and the fact that there is nobody is there to prod me to take the quicker solution. I don’t mean this to be a value judgment, just a trade-off that has occurred.

A key practice is to keep communication open and share knowledge. Even though there is not the constant banter, you can still quickly and efficiently communicate with the rest of your team. For the most part, I feel that we have been successful at that.

Another practice is discipline is even more important. We no longer have our pair to make sure we are following good practices. That means, we need to keep ourselves in check. That means slower development on each track, but more tracks should mean we get more done. We must also be disciplined enough to recognize situations where some code can be controversial and ask for the help of a peer.

Integrating our work is also of the upmost importance. We still perform TDD and have a continuous integration server. We have verification via a smoke suite and manual QA to ensure that we release correct software. We also practice shared code ownership, and try to communicate issues with code.

So in the sense, we have modified our system of software development to account for present realities. That is not to say that everything we are doing is correct, but if problems emerge, we can combat them. In a future, pair programming may be the best option, given different circumstances.