We’ve been kind of quiet lately because we’ve been working on some big new features. I can’t give you all access to them just yet, though you may have seen some calls for testers go out on Twitter. Testing new Scout Labs features is a great way to see what’s coming soon — tests usually take only about 15 minutes (and could get you a free t-shirt!) In the meantime, here are some enhancements to the application, released today.

• Easier bookmarking. It is now very easy to bookmark anything you see in the application on your initial viewing of it — just click the “Bookmark” link. You don’t have to click through to the detail page. This makes creating lists of interesting links for reports and such SUPER easy.


• Links to Google Blog Search and Twitter. Now you can see results for Scout Labs searches on Google Bog Search or Twitter with a single click. This is really handy because you can just set up a persistent search in Scout Labs and not have to try and replicate the same query on Google or Twitter to check across sources. If you find an interesting mention on Google that you’d like us to have in the Scout Labs application, to score sentiment on etc, then feel free to add it using the “Suggest a Source” link at the bottom of every page in the application.



• Pagination for media sources like Flickr and YouTube. Now you can page through all the results from photo and video sources like Flickr and YouTube, and have the same collaboration features active on all of them — bookmark them, email them, discuss them, and so forth.


• Turn off billing notices for admins on an account. If you are an admin but don’t want to receive billing notices for your account, just go to your profile and “Edit” to get to the profile setting that turns them off.


• Added permalinks to all email alerts. Now it is easy to get right to the content we’ve emailed you about from your email alert.

Keep the feedback coming — we always want to know what data/features are most important to you!

We’ve been monitoring the Iran situation within Scout Labs for the last few weeks and its been very interesting to see the memes of conversation over even such a short period of time. Last week, in our “Iran” search, the new memes of conversation in the Frequent Words module included:

• riots
• twitter
• election

This week, “riots” has been replaced with “protests”. Twitter is a top meme, but it’s old news (no longer orange.) The top new emerging memes in the last 24 hours include:

• Obama
• condemns
• political
• video

(referring to the Youtube video of the woman being execution on the streets of Iran that even Obama said that had indeed watched.)

But a new word has appeared that we have not seen associated with the search “Iran” before: “HOPE”
Let’s hope we’re right.

It is with great sadness that we acknowledge the death of Matt Ericson, one of the Scout Labs software engineers. Matt had been battling cancer since last fall, when he fell acutely and mysteriously ill on a vacation to Europe and very nearly did not make it home to be accurately diagnosed. We were grateful for his survival then, and hopeful that since the type of cancer he had (lymphoma) was a treatable kind, with a high survival rate, that this would be an episode of ill health in an otherwise long and happy life.

Matt died last Thursday at Stanford Hospital, which he had been in and out of for treatment over the last couple months. I’m still in shock. My own mental defenses against the worst outcome are still coming down. Each time I remember they crumble a little further and I cry a little more and think about how an early and untimely death exposes all that’s wrong with the universe, especially when the person in question was so funny, and goofy, and kind, and talented, and in love. At least he had Kristen and he was so very, very happy with her. It shone out of him. Look at him in this hat she knit for him. I love this picture.

Kirsten is organizing a celebration of Matt’s life for June 25 or 26 in San Francisco. The details will be posted on Matt’s blog, http://cancermatt.wordpress.com.

Matt, we miss you so much. Safe travels, brother.

A reporter asked me a few days ago if I believe that traditional branding (TV ads, billboards, logos) is dead, to be replaced by conversations. Scout Labs is called a “new marketing” platform, and in fact, money for Scout Labs often comes from traditional marketing budgets. So you might expect me to be on the “traditional branding is dead” bandwagon.

But I’m not.

Humans are wired to be receptive to branding. In fact, the brain reacts in a unique way when it spots a famous brand name, according to some research. Consumers reading a brand name do not treat it like any other word — instead they activate parts of the brain normally used to process emotions, one study claims. We literally have an emotional connection to brands. And advertisements (done well) are stories, told around the flickering, mesmerizing light of the modern-day-campfire called television. We love stories. We always have and we always will.

If you want to now just how primal and powerful branding is for humans, look to children and the ease with which they swim in the vast branding sea. From the Institute of National Media and the Family, children as young as age three recognize brand logos (Fischer, 1991), with brand loyalty influence starting at age two (McNeal, 1992).

Now, my children are a tad more ad-savvy than most children. They don’t watch TV much (just movies) but I talk to them about marketing and encourage them to be conscious advertising targets. They talk about ads and marketing quite often, but this was a particularly fruitful weekend.

Saturday, Rowan (age 4) and I headed to our favorite Santa Cruz News Café on Mission Street for some hot tea and conversation with the locals and their dogs. Mission Street is the commercial district of Highway 1, so lots of trucks inch by the Café, which Rowan loves. Usually he points out wheels and axles and engines, but yesterday he had brands on the brain.

“Look mom, a turkey truck!” I looked up and had no idea what he was referring to. All I saw was a huge COLUMBUS (the salami) logo on the side of a delivery truck.

Then it dawned on me: “Rowan, did you remember that logo from the turkey we get at Trader Joes?”
“Yeah!” said Rowan.

Then a FedEx truck came by, idling as it waited for the green light. “Do you see the white arrow mom?” Rowan pointed. Many in the branding circles know of the brilliant, hidden arrow in the FedEx logo. I didn’t know 4 year old were primed to spot it. But I guess they are.


(It may be that his reading skills are not that strong yet so he looks at a logo and sees colors and shapes - not words, but still, it was pretty remarkable.)

Then, later that night, the whole family (dog included) piled into the truck with bikes and helmets in tow and headed off to Happy Valley School where the kids could ride bikes on flat, open pavement until dark and Sam and I could sit and talk between wipe-outs. En route, Fiona looked out the back window and saw a grey Audi following us. She immediately said, “Ah, those guys saw the commercial and got that car.”

“What commercial? What car?” I asked.

“They saw the commercial that says, ‘if you have an old house that isn’t that cool, kind of like Great Grandmas, then you should change your house and make it more like Auntie Jesse’s or our house. And once you change your house, you should change your dog and get a better dog. And then once you’ve changed your house and your dog, then you gotta change your car.’ So they must have fixed up their house, then got that car.”

Anyone who watched the opening ceremonies of the Olympics knows exactly what she’s talking about:

And she’s probably pretty accurate!

Our dear friend and colleague, Matt Ericson, lost his battle with cancer (Lymphoma) on Thursday.

He was a truly talented engineer and one of the happiest, most optimistic people I have ever met. He will be missed by the entire Scout Labs team. He chronicled the struggle for all of us in his blog, which has updates, now, from his wife Kirsten about the celebration in his honor that will be held later in June.

Matt, we miss you but feel so lucky to have known you.

PS: Please sponsor your ‘Team in Training’ friends or donate directly to the The Leukemia & Lymphoma Society.

Since the Scout Labs application began, we’ve maintained a rapid pace of development for new features. These new features make the application more powerful and usable, but only if they are bug-free. And since the nature of computer programming makes bugs inevitable, we have comprehensive tests to help us catch those bugs and correct them.

Once a new feature or change to an existing feature is completed and deployed to our staging environment, the next step is to write new test scripts to ensure the new functionality behaves as expected. All the older scripts are executed as well to regression test the existing features to make sure they are still functional.

In addition to the Java and Ruby on Rails tests, we do end-user testing. For this we use Selenium Remote Control, an open-source web app testing tool that works in multiple browsers and operating systems. It allows us to test multiple versions of browsers in different operating systems, including Internet Explorer 6 & 7, Firefox 2 & 3 on Windows XP, Windows Vista, Mac OSX Tiger, and Mac OSX Leopard. Selenium also supports Safari; we’ll be adding test environments with that browser soon.

When the Selenium Remote Control (SRC) server is running, it accepts commands from a script to automate browser actions. Anything from clicking a button or link to parsing the HTML on a page can be automated.

Using Selenium and a driver for Ruby, we have created a comprehensive test suite that simulates an end-user browsing the Scout Labs app (albeit at super speed). This way, we can test the functionality of the whole site, from simple to complex:

• Simple: when the app is used to send a blog post email, it should be sent immediately with the correct content.
• Complex: when a user creates a bookmark on a blog post, photo, video, twitter, or comment entry, that bookmark has the correct tags, the correct create date, can only be deleted by the author, and shows up in the correct places (on the entry detail page, on the search result page of the corresponding search, in the bookmark section, under the activity feed of the homepage, and in all other members of that workspace).

If the test suite finds a problem, major or minor, a developer fixes it and the test is rerun. Only when all tests have passed on all browsers in all operating systems is the new version promoted to the public site. This ensures we catch as many bugs as possible so you have a smoother experience in the application.

Of course, no test suite is perfect and Selenium can’t catch all the bugs. If something does break, you will see an error submission form; please fill it out and we will fix it as soon as we can!

Last night we pushed some new features that users were VERY vociferous about wanting:

• New data in the application: Blog Comments
• Export of data: Now you can download a list of your bookmarks, or .csv or .html files of your blog results
• Custom date ranges: You can set date ranges for blog data, so that you can see results from a particular time period

New data in the application: Blog Comments

Now you can see comment mentions for blogs, the same as you can for Twitter, Photos, Videos, Blogs, etc. This is hugely helpful when the main blog post does not contain a reference to your search, for example oDesk or Motorola or Dippin’ Dots, but the comments on the post do.

Export of Data: CSV and HTML

You’ve always been able to download the source data for graphs as a .csv or a .png. This new feature enables you to get a list of links with their content in either .csv or .html format, so you can work with them offline, excerpt from them for reports, or just read them through without having to click on anything (those were the three main use cases customers told us about). After you decide to download and choose your file format, the application emails you a link where you can pick up the data. Typically this takes less than 5 minutes. The limit per export file is 1,000 results at a time, but if you really want to read all 60,000 Coke mentions for the last 6 months, you can do it in tandem with the custom date range feature.

Custom Date Ranges: Set any range within the last 6 months

We’ve been showing you data within a 24H, 1W, etc range based on today’s date. Now you can decide what date you want the date range to start or end from. So if you want to see data only from February 15 to March 15, select 1M ending on 3/15/09 and you’ll get blog mentions published within that time frame only. Choose “center” on a specific date to see all the posts around a newsworthy or other buzz generating event; for instance if you know that Web 2.0 was from March 31st to April 3rd and want to know what Jeremiah was doing around that time, center the search around April 1st and choose a date range.

There are a couple of other minor enhancements, like cooler buttons that show state and other nifty Javascript enhancements, but those are better experienced than described. Coming up we have greatly expanded Twitter data, so that you can get graphable trends back a couple months, use full Boolean search on Twitter content, use date ranges, get sentiment and see frequent words on Twitter content. We are also working on an amazing feature that uses NLP techniques to extract customer quotes from user generated content. You’ll be able to see what people love, or hate, or recommend or wish for about a product or brand or company or whatever you are searching for. It’s SO COOL, we can’t wait to show it to you! Have fun and stay tuned.

As Margaret described in her earlier post about Sentiment, our sentiment scores agree with humans about 75% of the time, right out of the gate.

“Is that good enough?” people ask us. “It depends,” is the answer.

It is extremely powerful to have the system score hundreds of thousands of posts in real-time so that you can be alerted to potentially volatile issues and situations without doing any special scoring work yourself. For example, when Netflix logged in earlier this month, (or maybe when they opened their daily email alert from Scout Labs) they saw this data:

Pretty clear that customers were not happy about something on Monday March 30th. Drilling in, Netflix could quickly see that the decision to increase the price of its Blu-Ray DVD rentals was not a popular one! We correctly scored lots of unhappy posts.

We didn’t score every post correctly. I found one post that we scored as negative, based on the following phrase: “Netflix blamed the company for…” We thought it meant that Netflix the company was being blamed, but really it was Netflix doing the blaming. But in aggregate, Netflix gets a very good sense in real-time what customers are thinking, thanks to Scout Labs’ automated sentiment scoring.

And when you care enough to make sentiment the very best, you can always correct or override the sentiment values in the application, which updates the value only for you and your team. If you are an agency (especially PR) this is part of the value that you can provide to your client: getting the sentiment values JUST right. There’s an added benefit to scoring / improving sentiment, as the team at Scout Labs uses those overrides as labeled data sets that we can use to improve our sentiment accuracy rate over time.

So is sentiment scoring at Scout Labs perfect? No. Is it still incredibly powerful and useful in identifying problems? Yes. It acts as an early-warning system and brings very important problems to your attention.

Today we have two new features available in the application: six months of data and agency co-branding of workspaces.

Agency co-branding is the ability to add a second logo to the workspace header, with custom text that will appear on the homepage and on click for the second logo. Agencies have been asking for the ability to co-brand the application for clients that they share a workspace with, like so:

This feature will support the addition of any second logo to a workspace, for instance the iPhone group within Apple, or the Basketball team within Nike. Go to the “Settings” tab and note that there are now two slots for uploaded images plus a agency info/custom text field. Just make sure you hit “refresh” after you upload new images to see your new assets in the header. It is possible to format text using basic HTML or the Markdown syntax such as * for italics or ** for bold and to include links in the text field. More detail on how to format the agency info area in our Support section.

Six months of blog data is also now available. We are hearing from all of you that two+ years of data is optimal, to do year over year trending and get a little more history, so expect more data to be made available soon. Sentiment is available only for the past three months. Six month graphs are also available with one caveat: because we are only supporting sentiment on the last 3 months of data, six month sentiment view may flat line more than three months back. If you get more, consider it the gift of an idle 8-core processor!

Next up: Blog comments, one of the two remaining must-have data types (the other is message boards, which won’t be available until summer). Also customizable date ranges for viewing blog data and export of data, both of which have been repeatedly requested by users who want to tailor the view within the application and work with data outside it.

In terms of other major features, we are working hard on full Twitter data, including graphs and frequent word lists. You’ll be able to plug in a search and use more exact search parameters than are supported by search.twitter.com and get trendable graph data, which no one else has. We’ll definitely let you know as soon as that one is ready!

At the end of last week we released a new feature that allows users to show or hide spam results for a given search. Now you can either see your blog & news search results with spam suppressed (the clean view), or opt to see the ones Scout Labs or other members of your team have marked as spam (the dirt). Whether you remove the result entirely or mark it as spam, the mention will not be included in graph data.

Depending on your search, spam volume could be either a trickle (California Bureau of Land Reclamation) or a flood (Britney). Warning: A lot of the spam content for ANY search is pretty “unsafe” for work!

Of course most of the spam we find is just annoying. Take a look at this safe-to-blog-about spam example. For a general Nike search, there are ~2800 legitimate results and ~370 spam results. Check out the spam results. They look pretty darn spammy to us:

This feature is especially useful for:

• Marketers who need to know everywhere their brand is being mentioned, including the dubious mentions, but don’t need the spam mixed in with their “real” results


• Search marketers trying to figure out when their search traffic is being hijacked by sploggers, so they can bid down campaigns


• Analysts trying to clean up results to make their graph numbers as accurate as possible (put all those sketchy results in the “spam” bucket and you won’t see them in graph data)


• PR people who need to know every place their client or client’s brand is being mentioned, particularly the dubious mentions, in case these mentions require action to address

Of course part of the reason we released this functionality is that we have received a lot of positive feedback about our spam reduction measures, so we really want to keep improving it—and the more results that our users mark as spam, the better our training data will be.

In next week’s release expect to see:

• 6 months data available in the application (sentiment processing will still be only for the last 3 months, alas—if you want more, let us know at support <at> scoutlabs <dot> com or via Twitter—so we have an idea how important more sentiment history is to you


• Agency cobranding for workspaces: the ability to put up two logos, a main one in the top left of the header, a secondary one in the top right of the header, and add some custom text to the homepage. This is for all you agencies out there who are using the application on behalf of a client and want to share the workspace with them

In general we release code with new features every week. It’s not always a major feature and this past few weeks since launch our releases have been mostly commerce and account features that work through launch kinks (More of you want to pay via invoice than we thought! Who knew credit cards issued in Australia wouldn’t like a 1 cent validation!), but there’s a bunch of more interesting content and features coming up soon. More on our roadmap forthcoming.