Flaptor

Right now there is a firehose of information on Twitter about the situation in Iran. For the past week, between 1% and 3% of all tweets have been tagged with #iranelection.

Boing Boing published a Cyberwar guide for Iran elections
which tries to clarify the confusion, but point #4 on it adds to it:

4. Help cover the bloggers: change your twitter settings so that your location is TEHRAN and your time zone is GMT +3.30. Security forces are hunting for bloggers using location and timezone searches. If we all become ‘Iranians’ it becomes much harder to find them.

This just doesn’t make sense. Security forces do not have a magical Twitter search engine that allows them to find people by anything other than hints people themselves give about their location. Let’s explain this better:

Put yourself in the place of the security forces. Let’s assume that they have people who understand the nuts and bolts of the Twitter API. How would they go about finding people? For one, you cannot search by timezone. Changing your timezone to Tehran’s doesn’t obscure any searches. You can search by location. If you are in Iran and you have a cellphone that constantly updates Twitter with your latitude and longitude, you could be found. Of course, you would only enable that information if you wanted to be found.

Let’s assume a person in Iran is twittering surreptitiously and doesn’t want to be found by the police. They may want to change the location on their profile to Butte, Montana and their timezone to US Mountain. This is not much of a security measure though. Suppose you are twittering from New York and your profile truthfully discloses your location. I’m interested in finding you because of the things you say. Knowing that you are in New York doesn’t help me in the least. If on the other hand you say that you are the tall guy wearing the Knicks jersey next to the Washington Square Arch, then I could find you. Again, why would you say that if you didn’t want to be found? If I wanted to find you and you didn’t want to be found, I’d have to track everything you said hoping for a slip that gave me a useful hint as to your whereabouts.

If I were a blogger/twitterer in a surveillance society, I would assume that a computer at an Internet cafe is not safe. It may have a keystroke logger and it could be connected to some central agency whenever certain keywords are typed. I don’t know if that’s the case in Iran, but I would not risk it. Likewise, ISPs could be monitoring traffic. If that were the case, everyone else outside of your country could pretend to be you on Twitter and it wouldn’t help you in the least.

Let’s assume you are twittering from a secure computer, using a proxy and your traffic is not being monitored by the government (either because you are either using encryption or because they just didn’t get around to it). If that’s the case, make sure that you cannot be personally identified/located from your information and you’ll be fine.

For everyone else, you can support Iranians by making your avatar green, changing your location (and your timezone if you will). Remember it’s all symbolic though. You are showing your solidarity and that’s good. Just keep in mind that you are not helping anyone avoid the security forces.

We are happy to announce our new release of Twist. These are some of the new features:

- Charts for pretty much every concept. For concepts that are not very popular or even obscure, we still draw a near-real-time chart with recent tweets. For example, see the trend for Flaptor or try your own website/brand/Twitter id and see how often it’s mentioned!

- More granularity and zooming in. All charts now have hourly granularity, even the 30-day view. Select a portion of the chart with your mouse and zoom in to see the only a few hours from a given day, or a few days from the monthly view.

- Better and more frequent calculation of hot concepts. Come back often to see what’s hot this hour and why.

- Last 24 hours chart. Check it out:

More features to come soon, stay tuned.

We just launched a new version of Twist with a number of new features:

- Updated look and feel. The page now displays percentage values (% of tweets that mention the concept) and highlights the intervals during which the concept was hot.

- Zooming in to a portion of the chart (by clicking and dragging the cursor) and getting information for a specific time (single-clicking a point in the chart).

- Context: we added a pane that shows tweets about the concept at a given time (peak or not). Also, when possible we display a brief explanation/definition of the topic (retrieved from Wikipedia) along with a link to the article.

- Browsing all hot topics from the past seven days. As hot topic detection is probably the most interesting and useful feature of Twist, we put all the hot topics in one page. There is a daily/weekly view and much more detailed information for each topic. Also, you can see what was hot at a given moment. When clicking on a point in the chart, the new pane shows other topics that were hot at that time.

- Better embed. It is now possible to embed a dynamic or static chart, choosing between two different sizes.

There is more, take a look at Twist’s Help and New Features.

A post on Techcrunch questions Marissa Mayer’s assertion that “search is 90 percent solved”. There is an interesting discussion in the comments about where search is going, but there is something more fundamental that is not addressed. Let’s leave the 90% progress mark aside for the moment. What does it mean for search to be solved?

In order to solve a problem, we must first know what the problem is. How is “search” a definition of a problem? For many technologies it’s relatively easy to find potential improvements that could be considered problems to solve. As an example, let’s look at cellphones.

It seems obvious that there are many limitations of cellphones that stand in the way of useful applications. Battery life is an obvious one: an ideal cellphone may have a battery that lasts as long as the device itself, pretty much like a wrist watch. Perhaps it could be powered wirelessly. The point is that the user would not have to worry about limiting usage until the next recharge. Also the ideal cellphone would be extremely light, it would have a faster user interface and a better input mechanism, etc. It’s even easier to analyze the limitations of cars: lower prices, self-driving capabilities, better safety features are some issues that come to mind.

Now let’s think about Google’s web search. If we look at the problem of indexing all the text on the web so that a reasonable person can find a given document, how much better could it be? It’s already free for the end user, so “cheaper” is out of the question. It’s also hard to imagine a faster search, as Google returns results within a fraction of a second. In terms of coverage, there is a possibility that Google won’t find a document that is out there but its crawlers have not discovered yet. They have gone a long way in this respect and today it’s hard to make the case for the usefulness of a document that Google has not discovered within 24 hours of its publication. As for relevance, sometimes it can be hard to find a document that Google has indexed even after a reasonable number of tries with different combinations of keywords. I know this has happened to me in the past few months but I can’t remember a specific example (I think it was some product with an extremely generic name). Still, in my experience it’s very rarely an issue. In conclusion, for a certain definition of the problem it’s reasonable to state that it’s 90% solved.

Of course, it’s possible to describe a whole class of problems (many still unsolved) that can be labeled “search”. We mentioned some of them a while ago. Search for extraterrestrial life is an extreme example.

The main point of this post is one that happens all the time in software: someone creates a solution to a perceived problem. This solution uncovers other needs or problems. There comes a point when people lose track of the problems and it becomes very hard to discuss solutions. We can’t argue whether search is solved or not. Let’s understand and describe each specific problem first.

Thanks to al3x and ev from Twitter, Twist once again is receiving a significant volume of tweets. This means that in a few days it will be possible to see more detailed weekly trends and hot concepts. It’s already better for intra-day trends, check out the resolution for the past 24 hours:

We have been keeping track of the public Twitter timeline for a few months now. Until a few days all Twist showed was a 7-day window, but now you can select monthly and historical trends. For example, it seems that in late April people start talking more about the summer than about the spring:

Also, you can see weekly patterns. Check out how weekend is mentioned more around Labor Day Weekend.

Stay tuned for more updates. As always, any feedback is welcome!

Web apps are a lot like TV shows. In particular those that are supported by ads, because their source of revenue is the same and the viewer/user is not really the customer but the product. These days it seems that everyone has some advice for Twitter about possible business models. I really don’t think the folks at Twitter need to worry about that, if they don’t find one that works they should just focus on making the company a good acquisition target.

If you come up with an idea for a TV show and even put together a pilot, nobody asks what the business model should be because it’s obvious. A TV show is made to be sold to a network, who will in turn monetize it with ads. Why can’t you just skip that step and monetize it yourself? Simply because you don’t own the infrastructure required to broadcast a TV show. You could try to build it, but clearly it doesn’t make sense if all you have is just one show.

Now look at Google, Yahoo and MSN as the multimedia networks that they are. They have a system in place to monetize many of their offerings. Plus, they benefit from the economies of scale. Today you wouldn’t start a company to build something like Yahoo Finance. I don’t know how much revenue Yahoo Finance generates, but it doesn’t need to be much to be profitable given that the incremental cost for Yahoo to keep it running isn’t very high. Gmail is probably not very profitable but it’s still very useful to Google (brand awareness, data mining, playground for new ideas, etc).

What could a company like Google or Yahoo do with Twitter? In a way Twitter has a “search” component to it. You can ask people a question such as “what do you think about X?” and if you have a few followers you’ll probably get some answers that may start a conversation to refine the “results”. You cannot do that with an automated search engine. I’m sure it’s possible to use your recent tweets as one more input to give you better search results and show you more relevant ads (not necessarily on Twitter but on other applications). Also, there is a lot of information worth mining in Twitter but you don’t need to acquire the company to extract it :)

In short, building a product to sell to one company is a valid business model. It’s risky, like creating a piece of art that perhaps nobody will like. You only have one product that you can sell to at most one client. As long as you understand the risk and have investors who believe in your idea, there’s nothing wrong with it. Perhaps it’s a good time for large companies to start thinking of new applications like Television pilots. They should encourage small teams with good ideas to work on “pilot” web applications (like what Ycombinator does, but perhaps with a larger budget). If they like the “pilot”, they already own a chunk of it and can acquire the rest.

When I think of a successful web application, I picture one that has millions of users and runs smoothly (it works fast and it does not crash). Of course, very few are like this. Perhaps the first example that comes to mind is something like Google Search. In the following chart, Google Search is in the deep south and far east:

The above chart depicts four possible stages of the life of a web app. My guess is that the vast majority of all web apps never leave the northwest. Someone throws some php (or Ruby, whatever) together with a database schema and has a proof-of-concept up in a few hours or days. In many cases this is just fine. Maybe your boss wants some internal tool to keep track of inventory, or perhaps you have an idea that you want to show potential investors to see if it’s worth pursuing.

Some applications are in the northeast. This is usually a bad place to be, because it means that you spent a lot of effort creating a highly scalable application. Perhaps you and your team had long architectural meetings in which you imagined how you would take over the world. Your app would be distributable, you came up with an efficient database design and perhaps a caching mechanism for when you get dugg or slashdotted. As it turned out, you built it and they didn’t come. The uptime command on your server shows a system load of 0.00, and you could invite all your site’s visitors to have dinner at your place.

In the southwest quadrant we have applications like Twitter today, or Friendster a few years back. Because these applications need to grow more or less proportionally to the square of the number of users, things can get out of hand pretty quickly. Regardless, this is a good problem to have, as moving from the southwest from the southeast tends to be easier than moving from the north to the south. You’ve built something that people like, so you may be able to convince others to give you the funds you need to organize the move.

Of course the chart is a continuum. Twitter is still far from the southernmost point, and perhaps it will never get there unless the move to the east goes well. Where is your web app on this chart, and how do you plan to move to the southeast?

Twist got a lot of traffic today, as it made it to the top of del.icio.us/popular. Here are very positive comments from del.icio.us users. It’s fun to look at the logs and see what people are comparing. Also, moving it to Amazon EC2 was a good idea as the traffic is not a problem at all.

We just released a fun tool called Twist. If you are familiar with Google Trends, it should be obvious what it does. It allows you to track trends on Twitter over the past week, with a granularity of a couple of hours. While testing the tool we saw spikes in the charts whenever an interesting story broke (especially about technology or celebrities). We discovered trends such as the fact that people who Twitter tend to have lunch rather than breakfast on weekdays, but both are mentioned equally on weekends:

http://twist.flaptor.com/freq?gram=lunch,%20breakfast

This tool is coupled with our Twitter search engine, so you can see what people are saying about the concepts shown in the charts. We really like Twitter and we are excited about this project, so stay tuned for more improvements.

By the way, both applications run on Amazon EC2.