StyleFeeder Tech Blog

Some of us spent the day doing some mockups of some new StyleFeeder features with Balsamiq. I’ve done enough work with Adobe Flex to be slightly skeptical that it might have the polish enough to survive as a full-on desktop application, but there simply doesn’t seem to be any other good mockup applications out there. I have Illustrator and Omni Graffle on my laptop, but those are overkill. Illustrator is too industrial-strength to be used to put together some something as trivial as mockups. Graffle is great and I use it somewhat frequently, but I was looking for something ideally that was cross platform (I’m on a Mac while everybody else is on some flavor of Windows).

Since some of us have heard good things about Balsamiq recently, I figured it was worth a look, especially since their examples look fun and easy. The installation didn’t work properly and froze up Safari, Firefox and Camino until after I’d rebooted. After that, things were… well, great.

Savage and I made many pages of mockups today using this tool to further develop some ideas that have been sitting on a whiteboard for a few days. During the course of this exercise, a number of questions and ideas came up, so it was really worthwhile, especially because we were able to focus on the ideas without getting stuck or sidetracked with the mockup tool.

The only thing that I found myself wanting with Balsamiq was a Lorem Ipsum text generator to fill out a paragraph of sample text, but that’s only available in the pre-release of the next version. I’m glad to know that it’s coming soon, though.

Balsamiq itself is full featured enough to actually do work with, but doesn’t have so many features that it causes you to spend hours perfectly aligning bulleted lists or other UI components. And the output looks like a mockup – there’s no danger of mistaking it for a final product! (I can’t tolerate hearing “… but that’s not what it’s going to look like in real life, is it?”)

Balsamiq’s UI is good and it’s clear that Giacomo paid a lot of attention to detail. I noticed subtle things like a contextual copy/paste behavior that changed what was being pasted onto the canvas. If you open up a third party application and select some text and copy that to the clipboard, but then copy a UI widget within Balsamic, you can then paste the original text into it when you start editing the text on that control. It may sound hard to explain, but it was exactly the response I wanted from the application.

Balsamiq isn’t a lot of money: just $79, although we could have had it for free. Some of us contribute to open source projects, which qualifies us. But if you write nice blog posts about it, they’ll also give it you. Now, while I do appreciate the offer, I really object to the idea of reading a nice blog post and wondering if that’s effectively a PR placement or if it’s authentic (in the case of this writer, I assure you that we paid for our copies, so what I’m saying is not tainted). While I don’t have a problem with people getting Balsamiq for free in exchange for a bit of word of mouth marketing, I think that the blogger should be required to disclose that they got it for free (are you listening, Giacomo?).

There were a few things that I’d like to see in the future, though:

1. Printing – I wanted to print out the mockups and tape them on the wall so I could stare at them, but you can’t print from Balsamic! So I had to export them all as PNG files and then print from another graphics application.
2. I’d like to be able to open multiple files at a time.
3. Diagonal lines – you can’t draw them. I wanted to do this at least twice.
4. Unlocking locked components requires a trip to the “Edit” menu and wasn’t obvious at all.
5. Exporting a group of docs as a multi-page PDF would be super.

I think that Savage summed it up best after I asked him what he thought of it at the end of the say. “Well, it didn’t get in my way at all,” which is about as good a compliment as you can get from him!

Hear, hear!

Sequential programming is one of those things that is nice for controlled examples, but is starting to look awfully impractical in the real world.  In fact, we’ve had some discussions at StyleFeeder lately where we were talking about the things that we as a company could not exist without.  Open source software is one, for example.  Sharding is another.  Concurrent programming was also on the list.  And I don’t think that this is the result of a failure of imagination on our part, either: we really couldn’t exist without this stuff.

On almost any page on StyleFeeder, there are a few threads at work doing things like data retrieval from multiple data sources, recordkeeping, updating caches, putting messages on queues and the like.  We’ve made some wonderful little constructs for making this kind of work easy, the heart of which is a little component called Assembler.

You give Assembler a few FutureTasks and a timeout.  Like, “go and do these four things and just stop doing whatever you’re doing after 1000ms since it’ll be too late at that point”.  In addition to helping keep the application profile level when we are under load, it also dramatically speeds up the page load time.

Of course, we also rely heavily on Whirlycache and various message queues, too.  Both of those provide other benefits, but a solid understanding of concurrency is necessary in order to understand when you are seeing a problem and how to diagnose it.  Because, yes, we somtimes see funny race conditions that are really hard to reproduce.  In practice, you frequently have to be able to look at code and analyze execution paths to understand where things could possibly go wrong.

Experience with those skills is something that we all have – again, out of necessity.  Because I’m pretty sure that StyleFeeder can’t exist without threading at this point.  I’m eagerly watching Dan Milstein’s notes over at Lookery with his Erlang adventures.  Concurrency, indeed.

This graphic also available as SVG.

Let’s say you are a huge website with an application platform and API that other companies have invested many millions of dollars in.  You decide to improve that platform, great.  You decide to make it a major renovation rather than a series of improvements, not so great, but we’ll let that slide since the old one was so … quirky.  You set up the new version in a sandbox and let people migrate it gradually, great.  This new API makes unecessary changes to things like parameter names, not great at all.  You eventually decide that the new one is ready for prime time, great!  So you decide to replace the old API with the new API, and by replace you mean that the new one is where the old one used to be, complete with it’s renamed parameters and total lack of backwards compatibility, VERY MUCH NOT GREAT.

If you want to put out 2.0 of your API, you put it at a new place, and eventually shut off the old one if you need to.  You DO NOT pull a switcheroo without ensuring compatibility, just like you do not replace a 2nd floor window with a door unless you’ve had the foresight to put a floor on the other side.

Ricky Gervais vs Chris Martin. I’m losing productivity.

“Hello? Contegix?” “Good morning. We’ve observed an OutOfMemoryError on stylefeeder01. We’ve restarted the web application and it seems to be running fine.” “Thanks for taking care of this for us,” I’d say. At this point, I’d start digging through log files to find out what had happened.

We’ve certainly had our share of Java Garbage Collection fun here at StyleFeeder, from heap and PermGen exhaustion, to memory leaks in libraries we’ve used; we’ve even been smacked over the head with the poor choice of NewRatio default for the x86-64 platform. But, these new OOMEs we were seeing seemed especially odd. The (Hotspot 1.5.0_09) JVM would log a PermGen OOME (“java.lang.OutOfMemoryError: PermGen space”), then proceed like normal, running a Full GC shortly thereafter. The GC log would indicate that, yes, PermGen was temporarily full, but, no, there was no real lack of space—the Full GC had no trouble clearing up over half of the available PermGen space. It was almost as if the JVM was yelling for help before it even tried to clean up its own mess.

At this point, we were using the default server GC, the throughput parallel collector. The parallel collector has two types of cleanups: scavenges and Full GCs. When the young generation fills up, a scavenge is fired off, cleaning up the young generations of the heap. When the old generation or permanent generations fill up, a Full GC is run, which cleans up all memory areas under GC control. Or, at least, that’s my understanding of what’s supposed to happen. In our case, when the permanent generation filled up, the GC would first fire off an OOME, then run a Full GC.

This smelled like a bug in the JVM. We weren’t using the latest version of Hotspot and were a little hesitant to upgrade, so we tried something different: the mark and sweep garbage collector. We had been thinking about using mark and sweep to avoid the occasional long pauses that are characteristic of the parallel collector. So, we gave it a whirl, making sure to turn on the options that would do as much permanent generation cleaning as possible:

-XX:+UseConcMarkSweepGC -XX:+CMSPermGenSweepingEnabled -XX:+CMSClassUnloadingEnabled

We waited… and waited… and waited… At this point, it’s been three weeks without an OOME, so I think we can safely declare the change a success.

Here at StyleFeeder, we created a personal shopping engine built upon the idea of making online shopping easier and more personalized. Our personal shopping engine learns your style and delivers results that you’ll love in seconds. As you’ll see, this blog post serves as our official entry to the Forrester Groundswell Awards. Groundswell is a cool book that discusses harnessing the power of social technologies like blogs, social networks, and YouTube. We are all about tapping into the aggregate power of consumers at StyleFeeder, so obviously, we think these guys know what they’re talking about; Groundswell co-author Charlene Li has been very gracious with her time and advice as StyleFeeder has grown.

We wrote a bit about the idea of recommendations in a previous blog entry, but we’ll reiterate for convenience. If you’re a user of the StyleFeeder site, you’ve likely noticed a recommendations link at the top of every page. Click on it (if you are logged in) and you will see a list of products: your recommendations. If you’ve never rated anything on StyleFeeder, your recommendations still work, but they get better as we learn more about your style. At first, we’ll show you a list of the most highly rated products on StyleFeeder. Begin rating and updating your recommendations and something interesting things will happen: the products you see will start to become more relevant to your tastes. We use what is known as a collaborative filtering algorithm to determine your recommendations. By comparing how you rate products to how others rate them, we can find products you will like. The more products you rate, the better we understand your preferences and the better recommendations we can make…even search results will be personal to you.

You can easily add your personal StyleFeed to your blog, Facebook profile, MySpace page or social networking home so others can keep track of what you’re finding. Check out some short videos to see StyleFeeder in action.

All in all, let us know what you think. What you love or what you’d like to see in a personal shopping engine…? Feedback is much appreciated!

Name of entry: StyleFeeder’s Personal Shopping Engine

Category: Embracing

Description of entry:

StyleFeeder, the Web’s only personal shopping engine, is revolutionizing online shopping by enabling consumers to find what they want through the power of social data. The site takes advantage of the fact that existing search engines aren’t built to facilitate product searches; Google et al might work if you know exactly what you’re looking for, but what if you’re stuck at the category level (I need a new shirt or a new rug)? Delivering personalized results in seconds, StyleFeeder’s core functionality of personalized recommendations has been developed by Dr. Jason Rennie, a graduate of MIT. Since his Ph.D. work on recommendations, Jason has adapted recommendations technology to work on real-world data instead of toy data sets, and developed ways to crunch all of this data in practically real-time, which no other recommendation service is currently doing.

As shoppers rate items on the site, StyleFeeder learns about each person’s unique sense of style so we can help them find products they’ll like faster. But, why? Why are we doing this? How could a computer possibly understand “fashion” and “taste?” How can a web site know my taste as well as or better than my best friend? In a sense, it can’t. Our recommendation engine can’t “look” at an outfit and give you feedback. But, the engine has an advantage over your best friend: data, and lots of it. The engine can’t “see” the products that are added to the StyleFeeder site, but it has access to the opinions of thousands of users. Each of those users has expressed their taste by rating and bookmarking products. The recommendation engine can identify products you’ll like by comparing your ratings and StyleFeed to that of others. Whereas your best friend may only be able to browse a few sites a day, the StyleFeeder recommendation engine incorporates and filters thousands of new items daily, from mainstream retailers all the way to so-called “long tail” products created and marketed by individuals.

The site is also constantly on the lookout for new users whose style is compatible with yours: your StyleTwins. We’re not suggesting that you ditch your best friend, but the StyleFeeder recommendations engine gives you something that neither your best friend nor all the fashion magazines can provide: a product discovery service that’s personal to your tastes.

For fun, we also built a shopping application on Facebook that was designed to not only increase our userbase, but tap into people’s personal networks of friends on in a place where they’re already spending a lot of time – helping grow awareness and visibilty of StyleFeeder through word-of-mouth. We set out to create an app that was fun and viral but also useful – most of the functionality of our web site is accessible directly on Facebook,. Launched just last summer, StyleFeeder is, by far, the top shopping application on Facebook. We blew away the competion, believe me!

How does this entry accomplish business goals?

As with most Web 2.0 startups, our goals were simple — create a fun, useful personal shopping engine for people and capture as many users as possible. Oh yeah…and to revolutionize online shopping while we’re at it. Since StyleFeeder’s website revamp last year, the site’s growth has skyrocketed, aided by the shopping engine and its inherent viral appeal and ease-of-use. Check out some stats below:

• 900,000+ Registered Users (up from 20,000 in August 2007)

• 4.5MM Monthly Page Views (up from 500,000 in August 2007

• ~1.2MM Monthly Unique Visitors

• Largest Shopping App on Facebook (1.6MM+ Installs)

• 14,000,000+ product links on StyleFeeder, added by users and data feeds from top retailers and brands

• Extremely positive revenue growth and affiliate revenue growth; model calls for profitability by early 2009

Congrats to the respective party nominees:

According to recent data we’ve been collecting on StyleFeeder, not very.

It’s always been our contention that while major search engines may be effective if you know exactly what you’re looking for (like a Canon PowerShot 8D70I or a Free People Fleece Dolman Sweater), they don’t work for shoppers stuck at the category level. If you’re looking for a new shirt or coffee table but don’t have something particular in mind, you’re likely going to throw your hands up in frustration after wading through 10 or more pages of textual data. (This, by the way, is why we think starting your product search on a visually-oriented site that already knows your tastes and preferences is a no-brainer!)

We’ve recently been working to improve the experience of users who are discovering StyleFeeder through search engines, and were intrigued to find that 55% of shoppers are not finding what they are looking for using Google and the like. When natural search users land on one of our item detail pages, we show them a photo and description of the product the search engine pointed them to and ask a simple question: “Is this what you are looking for?” 55% of shoppers are answering “No”. Take bounce rates into account, and the percentage of people finding the right item through search engines may be as low as 20%!

We aim to gather more data in support of this theory, but our preliminary results suggest that there is lots of opportunity to help internet consumers shop more efficiently!

One of the KDD conference sessions I was excited about attending was “Data Integration.” I know, I know, it doesn’t have that sexy ring of “Social Networks,” but it’s a very real problem that StyleFeeder, as well as just about any company that stores gigabytes or terabytes of data, faces. In our case, we collect data feeds from numerous retailers, resulting in a combined catalog of fourteen million products. This leads to a number of challenges, especially in how to integrate the product entries. Each retailer uses a slightly different data format; titles, descriptions, tags, etc. are of varying quality; and there is no canonical identifier to detect identical products being sold by different retailers. From a machine learning perspective, it is a rich data set wherein lie a number of interesting problems. One problem I’ve already alluded to is duplicate detection—how to identify when the same product is being sold by different retailers. Since quality varies widely, we need to identify and remove entries of such poor quality that they may turn up incorrectly in search results (anomaly detection). Some retailers engage in “tag stuffing”; others include tangential or irrelevant description. So, in order to achieve good quality search results, there is a need to clean up the text entries, discarding sections which are unrelated to the product.

I was curious to see whether any of the KDD talks would address issues like these. The short answer is “yes,” but to a limited degree. The presentations tended to either focus on relatively simple problems (such as applying a set of rewrite rules to detect duplicate URLs), highly structured problems (such as automatic record linkage), and/or complex models which would be difficult to apply to a large data set. I didn’t see any work that dealt with data as unstructured and “messy” as ours. On the other hand, there were certainly some interesting ideas. Peter Christen presented a machine learning approach to linking records from disparate data sources. He proposes bootstrapping a Support Vector Machine using highly confident examples, thus avoiding the need for labeled training data. My vote for most interesting paper of the session was “Unsupervised Deduplication using Cross-Field Dependencies” by Hall, Sutton and McCallum. The key idea is that context can help determine whether two items are the same. For paper references, two conference abbreviations may be the same, but title words can be used to disambiguate. Similarly, two different products may have the same name, but description, brand and retailer can be used to tell them apart.