Main

October 25, 2006

Identity 2.0 - On how I can use george.chiramattel.com as my OpenID

This is a continuation of my previous post on OpenID.

Today I will explain, how you can personalize your OpenID URL when using the VeriSign Personal Identity provider (PIP) system.

Step 1: You should create a OpenID account with your Identity provider. In my case, my OpenID is http://georgeck.pip.verisignlabs.com. Now, that doesn't sound like a name I would want to remember or associate as my OpenID. Another reason why I wouldn't want to use the default ID created by my service provider is that, I don't want to be tied down to a specific vendor. What if my identity provider shuts down!

Step 2: Create a unique URL which you can control and setup a redirect to the Identity Provider. Again, taking my example - http://george.chiramattel.com is my site and I have full control over it. I edit the default page there and add the following meta info into the HTML header. (Remember the following example is for me and VeriSign PIP).

<html>
<head>
<link href="http://georgeck.pip.verisignlabs.com/server" rel="openid.server" />
<link href="http://georgeck.pip.verisignlabs.com/" rel="openid.delegate" />
<head/>

.. followed by the rest of the HTML document ..

This instructs the requesting site to redirect to my actual Identity provider. At a later point in time, if I wish to change my Identity provider, All I need to do is modify the above snippet and point it to my new provider.

October 23, 2006

Identity 2.0

First, let me define Identity 2.0. Wikipedia defines Identity 2.0 as....

Identity 2.0, also called digital identity, is the anticipated revolution of identity verification on the Internet using the OpenID standard. Identity 2.0 stems from the Web 2.0 theory of the world wide web transition. It's emphasis is a simple and open method of identity transactions similar to those in the physical world, such as driver's license.

OpenID is not the only player in this field. Other solutions include:

I also found a solution from whobar that

allows users to login to a website using InfoCard, OpenID or i-names and makes it all transparent to the web application

in which case the login screen would look something like this.

WhoBar

So, what does all this mean to the end user?

To understand this better, let me take you through the experience of authenticating myself to an OpenID enabled site. In this example I have used the free OpenID service provided by VeriSign called Personal Identity provider (PIP).

Step 1: I want to login to an OpenID enabled site. I am using Opinity as an example. To login, I am presented with a text box to enter my unique identity URL. Notice the OpenID logo in the text box.

Step-1

Step 2: Once I enter my unique ID, (in this case george.chiramattel.com), I am redirected to my Identity provider, which is VeriSign. I login to VeriSign and I am presented with the following screen from VeriSign. Here I can authorize VeriSign to 'prove' my identity to the referring site - Opinity. As you can see, I can set a timelimit to my authorization. I can also choose to share some personal profile information that I have previously stored with VeriSign with Opinity - but in this case Opinity is not expecting any such information.

Step-2

Notice that VeriSign mentions my name as http://georgeck.pip.verisignlabs.com. I will explain how I could configure this in a later blog. Once I confirm my authorization, I am redirected back to the referring site (Opinity) and (ideally) I should be logged in.

How is this different from a normal web-signin?

To start with, I don't have to maintain different passwords for different sites. My identity is my unique URL and I can present it to any site to let them know who I am. Now to prove it is really me who is presenting this URL, I use the service of the OpenID server, who ascertains my claim on my behalf.

I will explain details on how I configured my OpenID in a later blog.

September 30, 2006

With 'Dapper', you can treat the web like a database

What is Dapper? Well, according to the creators, dapper is...

... a service that allows you to extract and use information from any website on the Internet. For those familiar with web services, you can think of Dapper as an API maker. For the rest of you, Dapper allows you to build web applications and mashups using data from any website without any programming.

This means that you can treat the web like a database - well, almost. You can also call it a glorified screen-scrapper' with an Ajax frontend and a service to back it up.

A long time ago, I was inspired by an article with a similar theme. The title of the article was '.NET Web Data ToolKit'. You can find it here.

If you compare the two, with Dapper, we get the data published in standard, easily consumable formats like XML, RSS, JSON etc. With 'web data toolkit' (WDT), it is more programmatic and the results are returned as ADO.Net data sets. The author Tony Loton describes WDT as...

...a toolkit that allows you to SELECT nuggets of information FROM live web pages as though selecting from tables in a relational databas, using industry-standard SQL. Imagine if you could capture the results into an ADO.NET DataTable, allowing you to treat web data like any other data source.

Though this is a powerful idea, it has some drawbacks. To start with, we are dealing with some-one-else's data - it is after all screen scrapping! If you are OK with it and wants to take it to the next level - say, take any set of web site(s) and then mash them up to create totally new web services, then you stumble upon the next drawback. Which is the lack of a workflow.

In most non-trivial usecases the website exposes the data by taking the user through a sequence of steps. An example step could include a login, followed by a click on a link (say my account name) which than takes me to to my data (say my account details). So to get to the 'real' data, the user have to follow multiple steps and certain steps could be involve HTTP 'POST' and not get just simple "GET's. So the service should have support for executing a workflow of steps. On each step, we should be able to control the actual HTTP data being send to the server and also be able to specify alternate control path if things go wrong. So the support for a WorkFlow engine is critical.

The next problem with Dapper would be privacy. I cannot create a web service with Dapper to get to my personal data (say my bank details) as I would have to share my login credentials with the site. So what we really need is a toolkit that I can embed in my application - similar to Web Data ToolKit.

August 31, 2006

Migrated Folk-Mind.org to 'Google Apps for Domains'

I should admit that I was not able to spend adequate time on FolkMind. Now that I am back from my vacation, I am committing myself again to this. And this time I am hoping that I can maintain the momentum.

Google started an initiative called 'Google Apps for Domains'. According to Google, this service will..

..offer private-labeled email, IM and calendar tools to all of your users for free*, so they can share ideas and get things done more effectively. You can design and publish your organization's website, too. It's all hosted by Google, so there's no hardware or software for you to install or maintain.

One of the first steps that I did is to migrate the domain folk-mind.org to the above Google service. I should say that my migration experience was quite painless and was able to finish the whole activity in less than 2 hours (including adding the default home page).

My only crib is that, currently there is no option to change the 'A record' in the DNS. We can only modify the CNAME record. This results in folk-mind.org and www.folk-mind.org (this time with the www) returning two different pages. My brother helped me resolve this by using a web forward to folk-mind.org.

July 20, 2006

Web 2.0 tools to help plan and track my vacation

This is to list the set of web 2.0 tools that I am using to plan and track my vacation.

Google spreadsheets:

We are using Google Spreadsheets to consolidate all the data that we are getting from the web.

This is really useful when multiple people are simultaneously gathering data (my brother and sister-in-law who are currently in Belgium are also actively involved). With this tool, I can share my spreadsheet with multiple people and all of us can simultaneously edit the contents. This also helps in keeping the data always in sync.

Great tool!

Yahoo Trip planner:

Yahoo trip planner helps in trashing out the next level of details in the trip schedule.

The process starts by specifying the start and end date of the trip. Once this is doe, we can start to add meat to the schedule.

Yahoo also facilitates this process:

  • It lets you check what others have done on similar vacations
  • Helps you decide on what to do at different locations, by giving a list of things to choose from (with user reviews/ratings).
  • Also integrates with Flickr. So if you have uploaded some photos, this tool helps you associate different photo with different locations.
  • Helps in creating a trip journal.

I have just started my planning here. Again, this tool helps in keeping the plans made by all of us in sync.

Google calendar:

Though Yahoo trip will help you create an agenda for the vacation, if we need more fine grained detail, we need a real calender. That is where Google calendar comes in.

Google also lets you publish this calendar where ever you want. And of course, multiple people can simultaneously add to the calender.

I have published our calender at my site here.

technorati tags:, , , ,

November 25, 2005

Value of links

In my previous blog, I had talked about online currency and In this context I liked what Jeff Jarvis said.

In this new world, links are currency. Links grant authority. Links build branding. Links equal value.

November 21, 2005

You can speak freely, but they will hear you only if you pay!

Strange! But that is how the Internet works. There is practically very little entry barrier for publishing content (one's opinion) on the Internet. But unless you have the right currency or asset, the rest of the world will not be able to see/hear your what you have to say.

Let me explain this in a better way. For most people, search engines have become the primary entry point into the web. The result of a search is sorted based on relevance that the search engine associates with that particular content. Though most search engines have different ways of finding relevance, the most popular way of inferring this is by looking at what other Internet users have to say about the specific topic.

For newly published content, this will result in a catch-22 situation. Because the relevance is low, others users will not be able to find this. And unless they are able to find this data, they will not be able to vouch for it. The easiest way out of this situation is for someone (or site) with higher relevance (or 'page rank') to exercise their right to vouch for this content. This is exactly the currency that I mentioned before.

It is because of this need that sites like digg have become so popular. But even with Digg, the problem is not fully solved. I will explain this in greater detain in a subsequent blog....

November 13, 2005

Is FolkMind better than Google?

I have been discussing the idea behind FolkMind with one of my friend and the conversation went something like this:

Folk: So George, I have heard that you have coined this new term ‘FolkMind’. Tell me more about the idea behind it. I didn’t get time to go through the long blog that you have written on this.

George: It is about a new way of retrieving information from the Internet, using the collective intelligence of…

Folk: Oh, then that’s what it is, another search engine! Haven’t you heard about Google!!

George: Well not exactly. A good search engine will help you find information, but it doesn’t help you in understanding it. But FolkMind will!

Folk: What do you mean by ‘make you understand’?

George: Let’s take an example. Say you are searching for something like ‘Web 2.0’. Google returns you more than 15 million records.

Folk: Exactly you got the information that you wanted. It was that simple.

George: But that is too much information and like I mentioned in another blog, too much information is no information.

Folk: But the links that you got from Google are sorted based on the relevance. You know they use this algorithm called ‘PageRank’ which evaluates what the Internet community has to say about the subject being searched for.

George: To handle as huge an information source as the Internet, you require a totally decentralized community driven approach like PageRank. That’s obvious! But I still have to go through multiple pages to find out and understand what I was looking for.

Folk: Do you expect to find pages that exclusively contain what you are looking for?

George: Well, I don’t expect that! When you ask a question to an Oracle, you expect her to give a single answer instead of 50 different instructions, which you have to club together. Similarly, wouldn’t it be nice if the search engine could return you a page that is generated for me based on my query?

Folk: You mean an auto-generated page? Come on George, you must be kidding. I would prefer to read through 10 man-made pages to get what I want, than to go through all the junk returned by a bot.

George: Well, have you seen a newspaper that is made completely using an algorithm? The newspaper is called Google News!

Folk: Yes. I know about it. But news is different you know..

George: How come? As you would already know, Google News allows you to customize the news based on your preferences. So, in a way, it is also a search that you perform and the algorithm generates a page for you.

Folk: Hey, but Google News also just lists a set of links; you still have to navigate to the source page to make sense out of the information

George: That could be because Google doesn’t own the news and hence expects you to visit the original page. Well, coming back to the discussion on search, the result of a query need not be text that reads like a story; instead it could be a diagram that represents the essence of it. That way any bot that generates the result wouldn’t have to worry too much about structural stuff like grammar and readability. Think of the benefits – a picture is better for easy and quick comprehension as you would be using both halves of the brain when you ‘read’ a picture. Also, if you bookmark the picture instead of the text article, you will be able to recollect the matter by just glancing through it instead of reading the whole stuff again.

Consider the following facts:
 * 60% of what you read are structure words and not content.
 * 80% of the stuff that you read is forgotten within 24 hours.

Folk: What are you trying to say - that I am wasting my time in reading?

George: If your intension is to comprehend as much stuff in the shortest possible time, then reading verbose text is not the way for doing it. Say for example, suppose you were asked to review 50 code files. Would you start by reading from File1, File2 etc. or would you look for a UML diagram and then drill down for more details?

Folk: I would prefer UML diagrams and I agree that pictures are better than normal text for comprehension. But how can you compare structured data like code with text that you find on the Internet: You cannot create ‘UML’ like diagrams out of free text.

George: Yes. You are right. But if you do not want very precise data, why not represent the text as a MindMap. If you need more detail on any specific data you can always look at the original text by clicking on the appropriate section on the MindMap.

Folk: Hmm.. Sounds interesting. So, I do a Search and the result that I get is a MindMap that is generated by combining the related information from different places on the Web. What is returned is knowledge instead of links to thousands of sites where I can find the data.

George: Also the knowledge is presented in an easy to comprehend manner – as a MindMap.

Folk: But I know of a few other sites that do similar stuff, like Grokker and …what is the other name ...yes, Kartoo. How is FolkMind different?

George: Consider this. When you do a Google Search for say “Microsoft”, you get a few hundred million links. Anything beyond the first few pages are not useful for me. The tools that you mentioned above are designed to just solve this very problem. The value addition provided by these tools is that they can better represent the search results graphically. Grokker does a good job of categorizing the results. Kartoo goes one step ahead by also finding out common links between the results. So they are search enhancement tools. But as you see, FolkMind is more focused on retrieving “knowledge”.

Folk: Can you explain this better?

George: Sure. What you expect out of a search is not a list of probable links or their graphical representation. What you need is the “knowledge” based on your context.

Folk: Context? You mean..

George: When I do a Search, I am trying to extend my current knowledge along a particular axis. If a search engine really knows “me”, then it can return more relevant information. This would want the search engine to be aware of my current knowledge set

Folk: Hmm. I agree. This could be the reason behind the current trend of personalized search, search history and so on… so that the search engine knows my interests. So how does FolkMind satisfy this need?

George: To start with, FolkMind is a digitized version of the knowledge that you have. It helps you in extending you current knowledge by utilizing the Internet. The Internet is the biggest source of information and in a way represents the virtual brain of humanity.

Folk: A digitized version of my knowledge – what do you mean?

George: Let me take bookmarks as an example to explain this. When ever you find something interesting on the web you bookmark it. Right?

Folk: Yes – Obviously I won’t be able to recollect every thing that I have ever read and bookmarks help in recollecting stuff later on.

George: Precisely. Now assume that instead of bookmaking stuff, you use FolkMind to create a MindMap and store that instead of the bookmark. So when you have to recollect stuff, you can glance through the MindMap and be done with it instead of going through the whole text. Also assume that when adding new maps, FolkMind helps you in cross-linking it with previously stored maps. Over time this store will become a huge collection of interconnected concepts.

Folk: I can imagine. In a way this would resemble the human brain itself. Just like in the brain, we have neurons connecting to other neurons; FolkMind would have concepts connected to other concepts. Now I get what you mean by ‘digitized version of ones knowledge’!

George: The value of such a store will increase as the amount of data in it increases.

Folk: But George, that would be a lot of effort to create such a store. Do you think people will be willing to put in that extra effort?

George: Think of it like this. On one hand we have the extra effort to populate the MindMap and on the other hand you have the benefit of recollecting stuff easily. So if you reduce the effort in creating the MindMap store, the benefit could be enough motivation to put in that extra effort.

Folk: Well, you never know. Most people could be like me –Very lazy :-)

George: Ha Ha .. I know. That’s where the community can help. Take the example of Wikipedia. Only a fraction of the people takes the effort to update the articles. But many people use it.

Folk: How do you propose to reduce the effort for creating the MindMaps and that too using the community? I thought you just said that FolkMind would be a local application!

George: Think of FolkMind like Google Desktop. Yes, It runs locally but it can connect to the FolkMind server to get more data. Let me take an example. Assume FolkMind runs as a browser plug-in, always accessible to you. When you browse to a page, FolkMind can do some ‘Natural language processing’ on the text and will try to automatically create a MindMap and this is done in the background. Also the resulting MindMap is cached. The similarity with Google Desktop is that the former tries to index what ever you see; FolkMind tries to create a MindMap out of it. You can imagine that this auto generated MindMap would not be perfect. That’s where the user can help. If he is really interested in a particular article, he can invoke the plug-in and brings it to foreground. This will result in the MindMap graphically displayed over the article as an overlay. Also it presents tools for the user to quickly make corrections to the MindMap, like add, delete or modify nodes or connection. When a user bookmark this article, the MindMap thus generated is also linked with it.

Folk: Ok, now I understand how Natural Language Processing done by FolkMind can help reduce the effort put in by the user in creating MindMaps. How can the community help here.

George: This is the most important aspect of FolkMind. Any MindMap that is generated by the user is also saved on the server in his profile. The advantage of this is that wherever you connect, your MindMaps are always available to you. Also, FolkMind server can also share this data with other users. So the content that you helped create is used to increase the accuracy of MindMaps generated when someone else looks at the same page. Also you benefit from what others have contributed. This collective value addition is key to increasing the accuracy and relevance of the MindMaps generated. As more and more people use it the content becomes accurate and relevant. Over time more pages get stored and cross-linked.

Folk: WOW that’s a very powerful idea. FolkMind server can help create a Concept Map for the whole Internet. This will help change the way we look at the Internet. Instead of considering the Internet as a collection of web pages connected together using hyperlinks, using FolkMind we can visualize the Web as a collection of interlinked concepts. So, instead of navigating through hyperlinks, you can navigate through concepts.

George: This will also change the way you search for stuff. Instead of doing keyword searches you can do concept searches. And since FolkMind already knows about your context or knowledge set based on what you have already stored in your profile, it can retrieve you stuff that is very relevant to you or even suggest new stuff. It is like Amazon suggesting things to you based on what you have looked for before.

Folk: So if my knowledge set already contains stuff like Web 2.0 and AJAX, FolkMind can suggest to me things like Atlas as and when they emerge on the Internet.

George: Precisely. It can help you create new useful connections between existing concepts. In essence it can help you create new ideas!

Folk: Now I am really exited about this. How far have you finished this?

George: FolkMind is still in a very early stage of development. I have created a project on SourceForge. If you are interested you can join in. Also you can help spread the word by digging it here.

October 27, 2005

Registered Folk-mind.org

I have registered the domain folk-mind.org and is in the process of configuring this domain to point to the FolkMind SourceForge home. Though I would have preferred a domain name with out the hyphen in between, all flavors of this (.com, .net and .org) are already taken :-(

October 24, 2005

Too much of information is no information

I recently saw the movie PI, which is about a Math genius trying to figure out patterns in nature. I didn't like it very much, though IMDB users seem to have a different opinion.

One of the things that is said about the movie is that mathematicians have always tried to find repeating patterns in the expansion of PI and have come up with all possible sequences of numbers. This is another way of saying that there is no pattern in the expansion of PI (You won't believe it, but people have expanded pi to almost six and a half billion digits and you can get it from here).

To me this is similar to saying 'too much of information is no information'!; unless you have the right tools that can process all this data and present it in a meaningful way. This is very significant for the web which is the biggest source of information. Hence the need for better tooling - like FolkMind.

Let’s take News.com as an example. Instead of presenting users with a long list of new stories, they have added tooling that can help readers make more sense of the information and help them realize the connection between stories – In short help users realize ‘The Big Picture’.

So if we need to handle al the information that is available on the Internet, we need radically different tooling that can help us visualize the data in radically different way.

October 21, 2005

My two cents on Flock

Flock which claims to be a Mozilla-based "social web browser" has released a preview download.

To me one of the primary benefit of social software is that it can take advantage of the collective intelligence of the community or the 'The wisdom of the crowds '. In this respect Flock has not yet reached the mark :-(

I can tag and blog stuff from within the browser. Thus a user who is doing the above stuff (pursuing selfish interests) builds collective value for the rest of the users as a side effect. But the same user is not able to reap the resultant benefits.

An example of how a user can benefit from the generated wisdom is like Outfoxed, where the folk / community suggests better useful sites.

Another (better) example of a social browser could be FolkMind - which is ofcourse not yet build :-) But all are welcome to contribue to it.

October 19, 2005

Netvibes.com - An interesting Web 2.0 start page

According to them...

Netvibes.com is a customizable web 2.0 homepage solution.
This service is free and gives you the user the ability:
* to create a personalized page with the content you like.
* to put together data feeds and services from web 2.0 applications with a very simple interface
* to access your page anytime and from any computer .

October 18, 2005

Seamless Integration of Web Services

The following is an article that I co-authored (along with Brijesh Chenan and Rahul Sawhney) almost 4 years back. We represented Philips Software and presented this for the ‘Road Ahead Contest’ conducted by Microsoft India and got selected as the best paper :-)

But why talk about this now?
Well, I came across this document when I was going through some of my old papers and what struck me was that, most of the stuff that we predicted in this paper have (or is in the process of) become a reality. So please read on…

Introduction
Today we are witnessing another era of explosive growth on the web. The web has come a long way, from static content to dynamic content and now to programmable web services. The Internet has metamorphosed into a pervasive computing domain, providing value-added services. This means that the web is now available in ways never imagined in the past.

The web is now mature enough in terms of availability of services. All the functionality that you would want is probably already there on the web. But the truth is that only a knowledge worker can harness the full power of the programmable web. Further there is no integration of web services into applications that people use daily. It is time to start thinking about bringing this power to the common man.

This paper proposes some enhancements to the existing web services framework for better integration between web services and applications accessing them.

Need for Seamless Integration
While embarking on a web services approach, several important questions come to mind.

As web service providers,
* How can we make it easier for our customers to use our web services?
* How can we increase our client base?
* How can we ensure a viable business model around our web services?

As application developers,
* How can we make our applications more powerful by harnessing the power of web services?
* How can we provide our customers access to web services that are “yet to be developed”?
* How can we ensure that our applications are state of the art and provide the latest features without massive development costs?

The answers to these questions lead us to the next generation of killer applications.
These applications will be able to utilize the zillions of services out there and the ones that are yet to come. Armed with the knowledge of web services, they will be able to proactively help users in getting their work done. As developers we should focus not only on exposing services but also be capable of locating and binding to existing and future services on the net.

To illustrate, let’s look at the following scenario.

Scenario

Suppose you are interested in online trading and would like to check out the top-ten movers. Further, you might also want to transact on them. In today’s world, typically you would use your browser to search the web for online stock trading services. You would then choose one of these services and then navigate to its site. Here, you would click the link for the top-ten movers. In the list returned, you find MSFT [Symbol for Microsoft Corporation stock at NASDAQ] interesting and want to buy 100 shares. To do this, you would click the corresponding link on the site and make the transaction. Now, suppose you want more information about MSFT from some other service, or make the trade elsewhere, you would have to perform the same operations all over again at the second site. You obviously cannot use the data you obtained from the first site to invoke services at the second site.

Let us see how the killer applications of tomorrow alter this scenario. You fire up the next generation Excel and run a web query on the top-ten movers. The web query returns the results from the web service and formats it on screen. You find MSFT interesting, so you right click on the symbol ‘MSFT’. The context menu shows you the option “Web Actions -> Get Detailed Stock Quote…”. When you click this option, it returns the latest stock quote update on MSFT. You then decide to buy 100 shares, so you right click and select “Web Actions à Buy this stock…” on the context menu and go on to complete the transaction (See figure below).

Using Excel, you query your portfolio (probably from another web service) and see your open positions. You see that you are long on ‘MSFT’ and decide to sell 50. You go to the corresponding field and edit it to 50. Excel automatically prompts you “Do you want to sell?” You press “Yes” to complete the transaction.

Excel

How to realize this

This can be achieved through a combination of two enhancements to current web services framework.
* Mapping between data and web methods
* Mapping between common action and web service methods

Mapping between data and web methods

The WSDL [Web Service Description Language] file provides the application with an interface description of that web service. Each of these web methods returns XML data. With the current standards, it is not possible for the application to figure out what further can be done on these data. The application will have to rely on custom hard-coded logic to invoke further actions on the data. If we have another XML file that will contain the mapping between the data elements and what actions can be invoked on them, the application can be much more proactive in helping the user get the work done. These actions relate to other web-service methods, which could be provided by either the same or some other web-service provider.

In short we propose to provide a link to an XML file within the WSDL file for a web-service. Bodies such as W3C [World Wide Web Consortium] can standardize the schema for this XML file.

Mapping between data and web methods

With this in place, the application in the illustrated Scenario will be able to provide context actions like “Get Latest Stock Quote” and “Buy this stock” because it automatically knows what can be done with the data.

We feel that the concept of associating web services and data is very powerful. It opens up new avenues for applications to interact with multiple web services on behalf of the user without him having to write any programming logic.

Mapping between common actions and web service methods

We have seen how the mapping between data and actions will enable the user to invoke services for that data. But the user has to initiate this action, probably through a context menu. We can take this to the next level by making our applications intelligent so as to invoke corresponding services automatically.

In the above Scenario, when the user modified the field containing the number of stocks in his portfolio, Excel understood that the user is doing an ‘Edit’ action on that data item (which was returned by a web service). It is now up to Excel to find out which method corresponds to the ‘Edit’ action for the data item being edited. Once this method is identified, consent from the user is obtained to invoke it on the server to make the transaction.

Mapping between common actions and web service methods

A standardizing body can define an XML file that lists certain commonly performed actions (e.g. verbs like edit and delete). Next generation applications should be aware of these verbs defined in this XML file and be able to translate user-actions to them. A service provider while publishing the service can also provide a mapping XML file. This file specifies the verbs associated with data that is returned from each web method. In turn, each verb is mapped to a web method that is invoked when the user performs the corresponding action at the application side. The schema for this file should also be standardized.

Conclusion

The Internet will continue to evolve as a source of computational services rather than a repository of content. The web will gradually move towards a state where more and more value-added services will be presented as web services. The killer applications of tomorrow will be the ones that can seamlessly integrate these web services. These applications will replace the browser as the main interaction point to the web and blur the boundary between the local and the remote.

The standards of today focus more on how to locate, identify and bind to services. We should also focus on standards that define consumption of these services. This will allow applications to behave intelligently to identify and invoke services from disparate sources, transparent to the user.

You can also download the PowerPoint that was made for this contest here.

October 14, 2005

FolkMind – a killer app for the Web 2.0 era

Explosion of content on the Internet

Currently, a lot of information is being published in the Internet as blogs, which are rich in content, frequently published and scattered across numerous sites. MSDN Blogs alone hosts around 2000 blogs. It is very difficult to aggregate this information and get a consolidated view of the same. RSS readers and content aggregators have tried to help us achieve this to some extent, but these tools cannot scale up to handle the voluminous and widespread content in the Internet.

The brain as a better model for describing the Internet

The Internet in its current form represents the collective intelligence of humanity and it has many characteristics that are similar to the human brain. Just like new connections are made between neurons in our brain as we learn new things, as new concepts emerge on the Internet, the links between the documents that describe the concept gets stronger – quite similar to the organic growth that we see in the brain. Also in the brain the connections between the neuron are more important than the individual neuron (though the neuron forms an integral part of the whole, they are far too many to be considered individually). Similarly in the web the concepts that emerge out of connected documents are more interesting than the individual document itself. One might argue that a document(s) authored by a few authoritative person(s) on a subject might be a better representation than the collection of documents made by the whole crowd. Though it might sound counter intuitive, in his work titled 'the wisdom of crowds' James Suriowecki explains that a better representation and a more accurate picture emerges out of the collective decision of the crowd than the work of a selected few. This is also probably one of the factors behind the success of Wikipedia as an accurate encyclopedia.

Currently the most prominent means of getting information from the Internet is through search. This approach is good enough if we already know what we are looking for – or if we know the right question to ask. But if we visit the Internet with the intension of finding something new, a better approach would be to navigate the Internet directory or taxonomy.

Benefit of Folksonomy in favor of Taxonomy

Lets take the previous use-case of browsing through MSDN blogs, looking for something 'new’ and ‘interesting’ – this is a good example of a situation where I wouldn’t know what exactly to search for. Instead of individually going through each blog, I can extract the statistically unique terms to form a taxonomy. My intension in doing this exercise is to distill the contents of more than 2000 blogs into a few words and then pick out (from the resultant set) those terms that I find to be interesting. This can be easily done using TagCloud and the result is as follows.

MSDN-TagCloud.gif

You can see the original cloud here

Is this auto-generated taxonomy good enough representation of what is published through MSDN blogs? In my opinion-No. This result is purely statistical in nature and I would compare it to the result of Google index with out 'page rank'. It does not take into account the collaborative content selection and filtering that happens usually on the Internet. It is this additional data that makes the data more relevant. A better approximation would be to use the celebrative tagging also referred to as 'Folksonomy'. The term folksonomy is defined in wikipedia as "a neologism for a practice of collaborative categorization using freely chosen keywords. More colloquially, this refers to a group of people cooperating spontaneously to organize information into categories, typically using categories or tags on pages, or semantic links with types that evolve without much central control."

This kind of tagging allows for the kind of multiple, overlapping associations that the brain itself uses, rather than using rigid categories. Such flexibility in using tags is both good and bad. On one hand we have tags like 'blog' and 'blogs' appearing as different tags. On the positive side, a photo of a smiling baby might be tagged 'baby', 'happy' and 'cute'. So in effect folksonomy produces results that more accurately reflect the population's conceptual model of the information.

The need for better tooling

If the brain is a better representative model of how the Internet works then we need different kind of tools to navigate and retrieve information from it. To be able to cope up with the vast amount of information, it should be capable of navigating across concepts instead of across documents. And once we locate our exact match, we should be able to drill down to it. To explain this new UI, lets take the example of Google Earth. To be able to locate a particular spot on earth (which is not previously tagged by Google - hence not available to search), we can take two approaches. The brute force approach would be to hunt through all the locatable points on the surface, until we reach our point. A more efficient strategy would be to zoom out (in other words elevate our self to a higher altitude), where we get an over all picture and then drill down to our point of interest. So to handle more complex problems, we need to create better abstractions. Another benefit of higher abstractions is that, at higher level we can easily spot associations and connections between locations (or concepts) that are hard to find at ground level. In the analogy, just as we are able to navigate across continents, countries and states, we should be able to navigate across concepts that emerge out of the Internet. Another vital feature that is missing in the tools that are currently available is the ability to discover and make connections between concepts. It is this lack of tooling that led me to envision FolkMind.

The vision of FolkMind

To me the new killer app for the Internet should help me in working at any levels of abstraction. The higher the abstraction, the more volume of complexity and data I can handle. Also at any level of abstraction, I should be able to navigate between concepts that are visible at that level and observe new connections that were not apparent to me at a different abstraction. And when I want to dig deeper, it should help me in exploring more on that subject. At the lowest level of abstraction, it would resemble a browser. The mass of the content that is on the Internet will still be on HTML, which is doing a good job of capturing presentation information, and a browser is suitable to view this. In short this application should act as a seamless extension to mind and help me in generating ideas by creating new connection between concepts about which I have little or no previous knowledge by leveraging the collective intelligence of humanity. A 'mind map' would be a good UI (an example of such a UI is shown below) for representing the above-mentioned vision. Wikipedia defines the term 'mind map' as 'a pictorial representation of how a central concept is linked to other concepts and issues'.

As a start we can create a mind map of the existing folksonomy that already exists on the Internet (with data from sites like del.icio.us) and then add new content and nodes to it. In his article entitled "Using Wikipedia and the Yahoo API to give structure to flat lists", Matt Biddulph explains a simple method for automatically converting a set of terms into a connected graph. To me the idea of linking together concepts is quite powerful. Once we reach a critical mass of concepts defined in such a mind map, it can transform itself from a concept management tool to an idea generation tool.

brain_animation.gif

A new person who logs into FolkMind can start with the most popular folksonomy terms and from that point browse related concepts or he can start by searching for a particular concept. Each node in the mind map can be tagged with additional information like a short description, its relevance (based on algorithms similar in principle to the one used by Google page rank), and additional information. This is the highest level of abstraction and at this level the user is more concerned with the connections between concepts than the individual documents that contributed to that concept. With usage the folksonomy gets richer and more concepts and connections between concepts emerge. Once the relevant concepts are identified, the next step is to drill down to the individual documents that pertain to that subject.

If the Internet can be considered as the virtual brain that represents the collective intelligence of humanity, then FolkMind is the pictorial representation on the same expressed as a mind map – hence the name.

How does FolkMind fit in as a Web 2.0 application

Let me explain how FolkMind application demonstrates the traits that are commonly observed in Web 2.0 applications. FolkMind can be a thick client that connects to the FolkMind server to retrieve its content but it uses the client side processing power for rich interactive UI and for local cache. As the user interacts with the UI, any change is reflected back to the server (this is similar to how Google Earth works). Though a user is given the option of marking certain connections as private, all other new connections and nodes that are created by the user will be stored on the server and will be visible to other users. This action is similar to a person creating a bookmark and tagging it using del.icio.us. Thus a user pursuing selfish interests (the motive behind creating a new connection or node is for his own benefit) build collective value for the rest of the users as a side effect. This phenomenon (also referred to, as the network effect) is critical to the success of a Web 2.0 application. As more content gets added to the system and as more users join in, the value of existing users will grow.

Conclusion

As we have seen in this article, the vision behind FolkMind is to be a powerful application with an intuitive, interactive UI that can harness the power of Internet by being capable of handing huge volume of data. Eventually, this will become the virtual brain of humanity!

References

What Is Web 2.0 by Tim O’Reilly
Personal Brain
Using Wikipedia and the Yahoo API to give structure to flat lists

Tags:   

Updated: Fixed some typos and added a few tags

October 13, 2005

Added TagCloud to my sidebar

The following is a description of the service provided by Tagcloud...

TagCloud is an automated Folksonomy tool. Essentially, TagCloud searches any number of RSS feeds you specify, extracts keywords from the content and lists them according to prevalence within the RSS feeds. Clicking on the tag's link will display a list of all the article abstracts associated with that keyword.

I exported the blogs that I regularly read as an OPML file and provided this as an input to TagCloud to get this result. Now I can track the top 50 statistically unique things that are discussed on these blogs. Wonderful!

Archives

Subscribe by e-mail

Creative Commons License
This weblog is licensed under a Creative Commons License.