Social Computing: Trust, Reputation, Recommender Systems, and Social Networks

Patterns of File-Sharing in an Enterprise: Authors, Contributors, Collectors, and Lurkers

2009-11-12T18:51:00.000-08:00

Great work by Michael Muller of IBM

We describe Cattail, an experimental enterprise file-sharing service in IBM. Over the past several years, 17985 Cattail users have uploaded 132041 files, which have been used by 115538 users. In addition 15240 people have shared 75951 of the files with other users, and, 5444 people have created 12461 collections comprising 60476 of the files. We use this rich set of data to characterize file-sharing in the enterprise. This talk will describe Cattail, and the factors that lead to a file being of use to other people, analyzed in two timeframes: over the lifetime of a file, and within the microstructure of a user's session. We will also explore emergent roles within the file-sharing system, and we will conclude with a look at the work of lurkers in the enterprise.

Importance Algorithms by Jung

2009-02-27T08:00:00.000-08:00

>>> BaryCenter(someGraph)
A simple node importance ranker based on the total shortest path of the node. More central nodes in a connected component will have smaller overall shortest paths, and 'peripheral' nodes on the network will have larger overall shortest paths. Runing this ranker on a graph with more than one connected component will arbitarily mix nodes from both components. For this reason you should probably run this ranker on one
component only (but that goes for all rankers).
A simple example of usage is:
BaryCenter ranker = new BaryCenter(someGraph);
ranker.evaluate();
ranker.printRankings();

>>>BetweennessCentrality(someGraph)
Computes betweenness centrality for each vertex and edge in the graph. The result is that each vertex and edge has a UserData element of type MutableDouble whose key is 'centrality.BetweennessCentrality'.
Note: Many social network researchers like to normalize the betweenness values by dividing the values by (n-1)(n-2)/2. The values given here are unnormalized.
A simple example of usage is:
BetweennessCentrality ranker = new BetweennessCentrality(someGraph);
ranker.evaluate();
ranker.printRankings();

>>> DegreeDistributionRanker(someGraph)
A simple node importance ranker based on the degree of the node. The user can specify whether s/he wants to use the indegree or the outdegree as the metric. If the graph is undirected this option is effectively ignored. So for example, if the graph is directed and the user chooses to use in-degree, nodes with the highest in-degree will be ranked highest and similarly nodes with the lowest in-degree will be ranked lowest.
A simple example of usage is:
DegreeDistributionRanker ranker = new DegreeDistributionRanker(someGraph);
ranker.evaluate();
ranker.printRankings();

>>> HITS(someGraph)
Calculates the "hubs-and-authorities" importance measures for each node in a graph. These measures are defined recursively as follows:
The *hubness* of a node is the degree to which a node links to other important authorities
The *authoritativeness* of a node is the degree to which a node is pointed to by important hubs
Note: This algorithm uses the same key as HITSWithPriors for storing rank sccores.
A simple example of usage is:
HITS ranker = new HITS(someGraph);
ranker.evaluate();
ranker.printRankings();

>>> HITSWithPriors(someGraph,0.3,rootSet)
Algorithm that extends the HITS algorithm by incorporating root nodes (priors). Whereas in HITS the importance of a node is implicitly computed relative to all nodes in the graph, now importance is computed relative to the specified root nodes.
A simple example of usage is:
HITSWithPriors ranker = new HITSWithPriors(someGraph,0.3,rootSet);
ranker.evaluate();
ranker.printRankings();

>>> KStepMarkov(someGraph,rootSet,6,null)
Algorithm variant of PageRankWithPriors that computes the importance of a node based upon taking fixed-length random walks out from the root set and then computing the stationary probability of being at each node. Specifically, it computes the relative probability that the markov chain will spend at any particular node, given that it start in the root set and ends after k steps.
A simple example of usage is:
KStepMarkov ranker = new KStepMarkov(someGraph,rootSet,6,null);
ranker.evaluate();
ranker.printRankings();

>>> MarkovCentrality

>>> PageRank(someGraph,0.15)
This algorithm measures the importance of a node in terms of the fraction of time spent at that node relative to all other nodes. This fraction is measured by first transforming the graph into a first-order Markov chain where the transition probability of going from node u to node v is equal to (1-alpha)*[1/outdegree(u)] + alpha*(1/|V|) where |V| is the # of vertices in the graph and alpha is a parameter typically set to be between 0.1 and 0.2 (according to the authors). If u has no out-edges in the original graph then 0 is used instead of 1/outdegree(v). Once the markov chain is created, the stationary probability of being at each node (state) is computed using an iterative update method that is guaranteed to converge if the markov chain is ergodic.
A simple example of usage is:
PageRank ranker = new PageRank(someGraph,0.15);
ranker.evaluate();
ranker.printRankings();

>>> PageRankWithPriors(someGraph,0.3,1,rootSet,null)
Algorithm that extends the PageRank algorithm by incorporating root nodes (priors). Whereas in PageRank the importance of a node is implicitly computed relative to all nodes in the graph now importance is computed relative to the specified root nodes.
Note: This algorithm uses the same key as PageRank for storing rank sccores
A simple example of usage is:
PageRankWithPriors ranker = new PageRankWithPriors(someGraph,0.3,1,rootSet,null);
ranker.evaluate();
ranker.printRankings();

>>> RandomWalkBetweenness(someGraph) !!! undirected
Computes betweenness centrality for each vertex in the graph. The betweenness values in this case are based on random walks, measuring the expected number of times a node is traversed by a random walk averaged over all pairs of nodes. The result is that each vertex has a UserData element of type
MutableDouble whose key is 'centrality.RandomWalkBetweennessCentrality'
A simple example of usage is:
RandomWalkBetweenness ranker = new RandomWalkBetweenness(someGraph);
ranker.evaluate();
ranker.printRankings();

>>> RandomWalkBetweenness(someGraph,someSource,someTarget)
Computes s-t betweenness centrality for each vertex in the graph. The betweenness values in this case are based on random walks, measuring the expected number of times a node is traversed by a random walk from s to t. The result is that each vertex has a UserData element of type
MutableDouble whose key is 'centrality.RandomWalkBetweennessCentrality'
A simple example of usage is:
RandomWalkSTBetweenness ranker = new RandomWalkBetweenness(someGraph,someSource,someTarget);
ranker.evaluate();
ranker.printRankings();

>>> VoltageRanker
Ranks vertices in a graph according to their 'voltage' in an approximate solution to the Kirchoff equations. This is accomplished by tying "source" vertices to specified positive voltages, "sink" vertices to 0 V, and iteratively updating the voltage of each other vertex to the (weighted) average of the voltages of its neighbors. The resultant voltages will all be in the range [0, max] where max is the largest voltage of any source vertex (in the absence of negative source voltages; see below). A few notes about this algorithm's interpretation of the graph data: Higher edge weights are interpreted as indicative of greater influence/effect than lower edge weights. Negative edge weights (and negative "source" voltages) invalidate the interpretation of the resultant values as voltages. However, this algorithm will not reject graphs with negative edge weights or source voltages.Parallel edges are equivalent to a single edge whose weight is the sum of the weights on the parallel edges. Current flows along undirected edges in both directions, but only flows along directed edges in the direction of the edge.

>>> WeightedNIPaths(someGraph,2.0,6,rootSet)
This algorithm measures the importance of nodes based upon both the number and length of disjoint paths that lead to a given node from each of the nodes in the root set. Specifically the formula for measuring the importance of a node is given by: I(t|R) = sum_i=1_|P(r,t)|_{alpha^|p_i|} where alpha is the path decay coefficient, p_i is path i and P(r,t) is a set of maximum-sized node-disjoint paths from r to t.
This algorithm uses heuristic breadth-first search to try and find the maximum-sized set of node-disjoint paths between two nodes. As such, it is not guaranteed to give exact answers.
A simple example of usage is:
WeightedNIPaths ranker = new WeightedNIPaths(someGraph,2.0,6,rootSet);
ranker.evaluate();
ranker.printRankings();

Napoleon Dynamite Effect

2009-02-17T15:55:00.000-08:00

they can't seem to quite get to this 10% mark because of the effect of a the ratings on a certain subset of indie movies, the flash point of which is "Napoleon Dynamite."

The other polarizing indie-minded gems the programmers have noticed this same glitch with are: "Fahrenheit 9/11," "I Heart Huckabees," "Lost in Translation," "Kill Bill: Volume 1" (note, not volume 2!), "Sideways," and "The Life Aquatic with Steve Zissou."

here

Surviving the exaflood

2009-02-17T15:49:00.000-08:00

From The Economist: The internet: Predictions that an “exaflood” of traffic will overload the internet have been doing the rounds. But will it really happen?

Some notes on collaboration

2009-02-17T08:10:00.000-08:00

here

Less is More

2009-01-21T08:58:00.000-08:00

The Economist

Constant improvements mean that more features can be added to these products each year without increasing the price.

But now things are changing, partly because the industry is maturing, and partly because of the recession. Suddenly there is much more interest in products that apply the flip side of Moore’s law: instead of providing ever-increasing performance at a particular price, they provide a particular level of performance at an ever-lower price.

People do good to look good. Plus, don't pay them!

2009-01-21T08:22:00.000-08:00

The Economist

Question: what motivates people to give money to charities or donate blood, acts which are costly to the doer and primarily benefit others.

Answer: people do good in part because it makes them look good to those whose opinions they care about. Economists call this image motivation. Dan Ariely, Anat Bracha & Stephan Meier tested the importance of image motivation. if people do good to look good, introducing monetary or other rewards into the mix might be counterproductive (An observer who sees someone getting paid for donating blood, for example, would find it hard to differentiate between the donor’s intrinsic goodness and his greed).

Conclusion (for charities): Suppose, for example, that rewards were used to encourage people to support a certain cause with a minimum donation. If that cause then publicised those who were generous well beyond the minimum required of them, it would show that they were not just "in it for the money".

Weird (research)parallel: how to draw out more participation to human experiments by exploiting image motivation

Writing a Business Plan

2008-12-14T04:22:00.000-08:00

Sequoia's checklist

How Leaders Create and Use Networks

2008-11-13T11:50:00.000-08:00

How Leaders Create and Use Networks (pdf)

> She resolved to make time for an extracurricular passion—the theater—in a way that would enhance her business development activities. Four times a year, her secretary booked a buffet dinner at a downtown hotel and reserved a block of theater tickets. Key clients were invited. Through these events, Linda not only developed her own business but also learned about her clients’ companies in a way that generated ideas for other parts of her firm, thus enabling her to engage with colleagues.

> They network in order to obtain information continually, not just at formal meetings.

> they wait until they need something badly. The best networkers do exactly the opposite: They take every opportunity to give to, and receive from, the network, whether they need help or not.

Shall our research agenda revolve around Positive Psychology?

2008-08-21T14:19:00.000-07:00

Positive psychology is a recent branch of psychology that studies why some people feel particularly happy. "Positive psychologists seek to find and nurture genius and talent," and "to make normal life more fullfilling, not to cure mental illness" (wikipedia). The father of positive psychology, Martin Seligman, gave a talk at TED.

In his research, he has found that happy people differ from miserable ones in one respect: they have pleasant, good, and miningful lives (find below a rigorous description of what he means by pleasant, good, and miningful). He councluded his talk (at minute 20:00) by proposing to further research on how technology may make our lives pleasant, good, and miningful. Shall we enrich our research agenda? :-) It would be a good start to read Learned Optimism or Stumbling on Happiness.

-------------------------------------------

. Pleasant Life or the "life of enjoyment" examines how people optimally experience, forecast, and savor the positive feelings and emotions that are part of normal and healthy living (e.g. relationships, hobbies, interests, entertainment, etc.).

. Good Life or the "life of engagement" investigates the beneficial affects of immersion, absorption, and flow that individuals feel when optimally engaged with their primary activities. These states are experienced when there is a positive match between a person's strength and the task they are doing, i.e. when they feel confident that they can accomplish the tasks they face.

. Meaningful Life or "life of affiliation" questions how individuals derive a positive sense of well-being, belonging, meaning, and purpose from being part of and contributing back to something larger and more permanent than themselves (e.g. nature, social groups, organizations, movements, traditions, belief systems).

future: apple-controlled mobile browsing

2008-06-15T10:43:00.000-07:00

Imagine that iPhone 2.0 attains the kind of market penetration currently enjoyed by the iPod music player. ... We will have sleep-walked into a different world. How come? Because whereas the personal computer is an open, user-configurable device, the iPhone is a decidedly closed one. Nothing runs on it other than software approved by Apple. This is not because the iPhone is incapable of running complex software: on the contrary, it is, in fact, a powerful Unix computer. But users who treat it as such - for example, by installing 'unofficial' software on it - run the risk of having their device 'bricked' [disabled] the next time they synchronise it with the iTunes software on their PCs.

Of course anyone can write programs for the iPhone with the aid of Apple's (free) Systems Developers Kit. But the only way they can get them installed will be via Apple's 'App Store'. And nothing will get into the store unless it's been approved by Apple.

If Apple's strategy succeeds, an increasing proportion of internet users will access through a gateway entirely controlled by a single company.

From guardian.

Challenges & Drivers for Mobile web 2.0 applications

2008-06-06T05:17:00.000-07:00

This whitepaper categorizes mobile web 2.0 applications by three verbs - share, collaborate, exploit. For example, in location-based services, users share their location with others, collaborate with those narby and exploit local knowledge.

Challenges
* Telcos control the distribution channel and content on it.
-> It's impossible for small start-ups and developers to innovate
* Flat-rate data pricing undermines revenues
* Don't exist performance analytics for advertisers

Drivers
* Offer new ways to deliver ads
* Anywhere/anytime accessibility of mobile phones
* Convergence of mobile & web worlds
* Flat-rate pricing (cheaper and more transparent for customers)
* If operators' portals will be open --> each operator can search beyond its portal

Mobile Cultures

2008-06-03T06:43:00.000-07:00

It is tempting to look at Japan (and Tokyo in particular) as a vision of the future for other countries or cities. ... However, there are more fundamental aspects of a society that cannot be disentangled from their attitudes and integration of technology.

Firstly, Tokyo is all about public transport. Trains run at minute intervals and are packed. Public transport is prime info-snacking space.

Secondly, population density in Tokyo results, obviously, in small accommodations. One consequence of this is that people, and young people in particular, spend a lot of time outside". More.

Branding Yourself With A Blog

2008-06-03T06:35:00.000-07:00

" Certainly personal branding isn't a new concept, but the future of personal branding could be in at your fingertips with a blog. One of the first steps in creating a brand for yourself is to make your blog visible. Post meaningful entries, comment on your industry's top blogs, or simply gain a regular readership. Visibility creates opportunities" says Schawbel, a social media specialist at EMC Corporation. He believes that when you brand yourself, the competition becomes irrelevant. "The goal of personal branding is to be recruited based on your brand, not applying for jobs," Schawbel says. "

Seeing Our Signals: Combining location traces and web-based models for personal discovery

2008-03-07T07:07:00.000-08:00

Here is the paper.

By combining web applications and mobile phones, they are developing algorithms for implementing a Personal Environmental Impact Report (PEIR). PEIR is a mechanism for longitudinal documentation of both impact—what an individual does to the environment—and what the environment does to the individual, or exposure.

Eigen-Trend: Trend Analysis in the Blogosphere based on Singular Value Decompositions

2008-03-07T06:49:00.000-08:00

Here is the paper.

Situation: The blogosphere provides free large-scale information sources from which businesses can quickly learn trends (e.g., opinions and complaints from their customers). How? For example, by keeping track of how often relevant keywords are mentioned across blogs (summing up the occurrence of keywords).

Complication: This simple way of extracting trends has tree problems:
1) Different blogs contribute to the trend differently.
2) For the same keyword, different groups of blogs may have different interests.
3) Can we directly study and extract meaningful trends from such a dynamically changing blog graph structure? The blogosphere can be considered as a blog graph where the nodes are blogs and the links reflect endorsements and interactions among blogs. In addition, such a blog graph is changing with time as a result of the development of internal relationships (e.g., interactions among blogs) and external events (e.g., breaking news).

Solution: The authors propose eigen-trends, temporal indicators derived through singular value decomposition, that take differences among individual blogs in consideration. The key idea is to represent the observed data as a combination of information that captures temporal changes of the underlying data (i.e., eigen-trends) and information that captures the characteristics of individual bloggers (e.g., authority).

Boosting Topic-Based Publish-Subscribe Systems with Dynamic Clustering

2008-03-07T06:25:00.000-08:00

Great paper.

Problem: The maintenance overhead of a topic-based pub-sub system becomes particularly dominating when the system supports a large number of topics with moderate event frequency (e.g., news syndication scene).

Fact: One can typically detect correlations between users subscriptions, which can be used to group topics and reduce the overall maintenance cost. For instance, users that are subscribed to updates for a given piece of software (say the free bitmap image editor GIMP [25]) are likely to be also subscribed to updates of other software pieces on which the given software depends (e.g. GTK+, libart and Pango [9]).

Proposal: A dynamic distributed clustering algorithm that

+ utilizes correlations between user subscriptions to dynamically group topics together, into virtual topics (called topic-clusters).

+ continuously adapts the topic-clusters and, resp., the user subscriptions, to the changing system state by employing local cluster updates. Each local update is performed only when it is estimated to be (globally) cost effective. Furthermore, to minimize the overhead involved in gain estimations, a probabilistic component is employed to guarantee that (with high probability) gain estimation are computed only for updates that are likely to be beneficial.

Pecha kucha in Tokyo

2008-02-27T09:39:00.000-08:00

I went tonight, and it was really cool.

"We are all familiar with death by PowerPoint (one bullet at a time). Pecha kucha aims to cure this by limiting PowerPoint presentation to 20 slides, 20 seconds per slide. A total experience of just 6 minutes, 40 seconds. The idea is to build compelling presentations around powerful images and simple messages. "

Old article on Wired, and here are 3 presentations done worldwide:

| View | Upload your own

(try to) fly what you build!

2008-02-21T21:39:00.000-08:00

Patrick (my ex-flat mate in Tokyo) & his colleagues have designed a flight simulator in which you build your own plane. Watch it or give it a go.

State vs. Religion

2008-02-20T05:18:00.000-08:00

"Secular schools can never be tolerated because such schools have no religious instruction, and a general moral instruction without a religious foundation is built on air; consequently, all character training and religion must be derived from faith ...we need believing people."

This poorly-researched book makes points similar to the quote above, which is by ...

web 2.0

2008-02-03T16:29:00.000-08:00

Xavier Comtesse's matrix:
1- Knowledge
-- raw data
-- data with context
-- data with context over time that leads to
-- modelisation and then to
-- forecast

2- User involvement in the value chain:
-- receive info
-- pick and choose (self service)
-- customize (Do-it-yourself)
-- co-design
-- co-creation

HOWTO: Avoid common problems when writing a conference paper

2008-01-24T01:48:00.000-08:00

Tips by Peter Gutmann.

Every year I serve on the program committees of a number of computer conferences, which means that I get given a pile of papers to review for publication. Every year I see the same problems with submitted papers. And every year I decide I should write down some guidelines on things to look out for when writing a computer conference/journal paper. This year I've finally got around to it... The following is a list of the problems that most commonly crop up in submitted papers. Treat this as a checklist to apply to the paper you're planning to write, or have already written.

Existing Material -----------------
Providing a summary of existing work in the area covered by your paper is extremely important. This serves two purposes, firstly it lets the reader know that you're familiar with the field, and more importantly it lets you know whether what you're doing is useful and/or original or not. I don't know how many papers I've received that I've had to reject with a comment like "This is a poorly-done reinvention of a technique from the 1980s" (and no matter what topic your paper is on, at least one referee is going to tell you that it's a reinvention of something that Multics was doing 40 years ago). Many authors simply plough ahead with a topic without (apparently) performing any checking to see whether anyone has done this before. Others do a quick check with Google and leave it at that, however online archives typically only reach back a decade or so and are unlikely to tell you that you're reinvented something that first appeared four decades ago.

I can't over-emphasise how important this first step is - I've seen fifteen- to twenty-page papers killed in an instant because the authors never checked to see whether someone else had had this idea before them.

Approaching the Problem -----------------------
Experimental design can be a bit of an art form. As the saying goes, a problem well stated is a problem half solved. Although it's relatively rare to find the kind of biased metrics that are traditional in competitive benchmarking (when the goal is to prove that your system is better than the competition's), it's unfortunately too common to find problems like an inappropriate level of detail (either too much or too little), or poor or even no analysis.

The level-of-detail issue is an important one to get right. You want to present enough information that others can repeat the experiment, but not enough detail to overwhelm the reader. At the too-little-detail end of the scale, I've seen papers with results that would be impossible for anyone else to reproduce, either because the raw data isn't available (a relatively common problem that others have complained about), or because data acquisition details are insufficiently specified, or because the data-to-information transformation step(s) are insufficiently explained. The goal of a scientific process is repeatability and verifiability. This can be particularly important if the results are unusual or controversial. In a psychology paper published in the late 1990s with rather astonishing results the authors took the unusual step of having the work repeated by independent teams in the US and Europe, as well as making the raw data available for others to use (many did, and got the same results as the original authors). Without this level of detail, the very next research result could invalidate yours, and without repeatability no-one can demonstrate that you were correct.

At the other end of the scale are the papers that are decorated with various distractions like snippets of source code that caught the authors' attention and long-winded discussions of possibly interesting but not terribly relevant minutiae. Yes, it's interesting stuff, but it detracts greatly from the main flow of the work. If you're going to do this, put it in a separate section so that it doesn't distract from the main body of the work.

One other thing that you need to do is state your assumptions in advance. This is particularly important for papers in areas like networking and security, where the lack of a proper network performance model or threat model can make a paper almost meaningless ("assume a perfectly spherical elephant of negligible mass and volume"). In theory this could help kill a paper - I've seen security papers where the threat model is taken straight from la- la land - but in practice this doesn't seem to be much of a problem. Unrealistic models are a sure sign that your workflow was { design a cool algorithm, invent an artificial problem for it to solve }. A friend once told me about a paper he'd co-authored on a neat algorithm for which they had to invent an artificial problem for the algorithm to solve. The next year the same conference where they'd published it received three more papers on solutions to this non-problem (it probably has its own research field by now). So you can usually get these things published if you're careful about which conference you submit to. On the other hand if I'm refereeing something I'd prefer to see an honest assessment of the algorithm's value ("it's a cool algorithm and we wanted to share the design with others") rather than some totally bogus attempt at a justification for publishing.
While an artificial network or threat model isn't a killer, lack of any model at all is, because people are going to come up with situations that your design doesn't handle and tell you to resubmit once you've addressed them. Providing a model frames your solution and lets the reader know what it is that you're trying to achieve. It also means that reviewers will try and pick holes in your model rather than your design. (An unfortunate side- effect of this is that you can then design completely unrealistic models, see the previous paragraph).

Analysis of Results -------------------
One thing that can completely kill a paper is lack of analysis of the results. I've seen otherwise fine papers die in review because the "analysis" consists of the authors stating what's obvious from the graphs they present ("foo increases linearly") but never explain why foo increases linearly, or whether foo increasing linearly is a good or bad thing. More scary are papers where highly suspicious results (for example an O(n) and O(n^3) algorithm having identical runtimes) pass unnoticed. This is an indication that either the authors have been careless or that they don't understand the data that they're looking at, neither of which provide much confidence in a scientific paper.

When you present your results, you need to provide an analysis of why you obtained those results, not just a voiceover for the graphs and diagrams. If you have an O(n) algorithm but are seeing identical runtimes for n=100, n= 1000, and n=10000, you'd better have a good explanation for this. More generally, any significant anomaly in your results will need either further investigation or an explanation. One of the purposes of a paper is to demonstrate (and convey to readers) an understanding of what you're doing. Multiple unexplained anomalies can lead to the impression that this understanding is absent in the paper.

Even worse than an unexplained anomaly is a totally unnoticed anomaly. I once read a paper on crypto performance measurement that compared (among other things) the performance of DES with triple DES, where triple DES is DES iterated three times. The execution times for DES and triple DES were identical (!!), but the author's hadn't appeared to notice this.

A feature of many network protocols (and underneath the protocols, network hardware) is the extensive cacheing and buffering performed whenever possible to reduce latency and speed throughput. One conference received a paper whose results showed that the SSL/TLS handshake was about a thousand times faster than the IPsec IKE handshake. Now despite the best efforts of the IPsec design committee, the probably didn't manage to make IKE more than a few times slower than SSL/TLS. What had happened was that SSL has the capability to perform a re-handshake using cached data, and what the paper was actually measuring was the overhead of n IPsec handshakes vs. 1 SSL handshake and n-1 cached reconnects.

Another thing that's important when presenting results of measurements is to explain (and demonstrate understanding of) exactly what it is that you're measuring. The two protocols mentioned above, IPsec and SSL, both have numerous variations in their crypto mechanisms that can lead to vastly different performance results. For example the IKE handshake was originally designed by mathematical cryptographers who assumed that all crypto operations had an O( 1 ) cost. As a result, the original IKE public-key handshake design required no less than four (!!) very expensive private-key operations, compared to zero for the SSL client and one for an SSL server.

It's important in areas like this to both describe explicitly which types of operations your results are presenting and to either measure the standard, default operating mode or if you're using variant modes to make it clear to the reader that the results are atypical and not necessarily representative of normal performance.

(There's a lot more to say about this area such as techniques for analytical modeling, how to handle outliers, sensitivity analysis, and so on. This is covered in a number of standard texts, of which the most extensive is probably Raj Jain's "The Art of Computer Systems Performance Analysis". You also need to adjust for things like clock granularity, the overhead of clock sampling, influence from external factors like other network traffic or CPU load, and so on. For example relying on the operating system's ability to generate distinct timing data on a per-process level isn't going to help if another process is thrashing the CPU cache, which no amount of process- granularity timing is going to fix).

In terms of statistical analysis of results, papers tend to fall into two categories: Either very little, or way too much. Very little can work (it's an O(n) algorithm, end of story). The other end of the scale is rather more problematic. The result is a paper that's only comprehensible to another statistician, a variation of the "much data, little analysis" problem mentioned earlier. Applying Noodleheinz's strategy for quantum normalisation to yield a correlation factor of 0.167 with a variance of 7.31 to a pile of data may be a fine demonstration of your grasp of statistics (or perhaps a demonstration of your lack of grasp of statistics), but it may as well be written in Klingon for most of the audience (unless your audience consists of the attendees of a conference on network performance analysis). If you're going to apply statistical techniques, explain what you're doing and explain the significance of the results so the reader can understand it as well.

Diagrams and Tables -------------------
Some papers are severely let down by the quality of the graphs and tables that they contain. There have been plenty of books written on the graphical representation of data, so I'll just go through the main problem areas here. The three biggest offenders that I've seen are inappropriate use of graphs, inappropriate scales, bad labeling, and bad choice of units.

In a number of cases graphs are used where none are necessary. Graphing white noise is an extreme example of this. Graphs are intended to show a trend in data. If there is none, use a bar graph or a table, not a line graph. Similarly, if your graph has less than (say) five data points, use a table, or just put the data inline in the text.

When you're presenting information on a graph, don't try and overload it. In almost all cases a graph should present one set of y-variables (and by extension one set of x-variables). In some very special cases you can use two sets of y-variables, but you'd better have a very good reason for this. Any more than this and you're making your graph incomprehensible. In addition, don't try and cram more than five or six curves onto a graph, or you run into the same problem.

Use of appropriate scales results in data being crammed into one corner on one side of a graph, or all detail for 99% of the data being lost through an attempt to fit a single anomalous spike into the result. Look at your graph. If 90% of it is taken up by empty real estate, you need to consider an alternative way of representing it. Conversely, if 90% of it is taken up by ink, you've got a similar problem. A good rule of thumb is to try to maximise the information-to-ink ratio in your graph. There's something called the three-quarter-high rule which says that the highest point on the graph should be at least three quarters of the horizontal offset of the rightmost point, which is meant to avoid biased representations when using techniques like nonzero origins of the axes, but can lead to odd-looking graphs (I'll leave it up to you whether you want to use this or not).

Another thing to be aware of is that when plotting data resulting from random quantification of performance (so that if you were to repeat the experiment the results wouldn't be exactly identical), you should really show confidence intervals in your graph. People often don't do this and it doesn't seem to cause any problems, but you should at least mention the fact in the accompanying text if what you're measuring requires it. Obviously if it's some completely deterministic process there's no need to do this, and doing so will just clutter the graph or text.

Bad labeling is another problem, and can take many forms. I've seen text crammed in at odd angles, complex graphs with data represented in near- indistinguishable shades of grey, graphs where it wasn't obvious whether large values were better or worse, graphs where the axes were barely labeled (10 minutes? 10 K/s? 10 MB?), graphs where the units aren't obvious ("Time" and a scale of 1...10, which could be nanoseconds, minutes, days, or years), and everything in between. Remember, graphs are intended to visually represent trends in data. If this isn't immediately obvious from your graph, redesign it or replace it with a table.

Finally, something that besets both graphs and tables is the inappropriate choice of scaling units, so that everything ends up with strings of leading or trailing zeroes. Go with the most common scaling unit. Even if "seconds" is a nice basic time unit, if all of your graph labels or table entries are coming out as "0.00000x s" then it's time to consider using microseconds and not seconds.

Where's the Rest of it? -----------------------
A problem that crops up in some papers is that the authors construct an (often elaborate) demonstration and analysis of a problem without really providing a solution. If the problem is of the grand-challenge type (for example breaking AES encryption) then this is fine. Showing how to do this is enough to get you worldwide attention, and fixing it becomes someone else's problem. On the other hand if the problem is a TCP/IP congestion issue under certain network conditions then simply demonstrating that there's a problem isn't sufficient. Even providing a general solution ("congestion control is Good") isn't sufficient. If your paper has pointed out that there's a problem, you also need to provide the fix for the problem. The contribution in an AES- breaking paper is to demonstrate that AES is broken, but the contribution in a general paper is the solution to the problem, not the problem itself.

Referee Feedback ----------------
There's a reason why conferences and journals provide detailed feedback from referees rather just a credit-card style "accepted/declined". Someone once submitted a paper to a conference that duplicated (somewhat poorly) work that others had done a few years earlier (see "Existing Material" above). I sent it back with a reference to the earlier work and a longish explanation of how they needed to provide significant new material in order to make it suitable for publication. A few months later they submitted the same paper, unchanged, to another conference. Since they hadn't bothered to update the paper, I didn't bother to update my comments, and sent it back with exactly the same feedback. A few months later they tried again at a third conference, and the same thing happened. I guess eventually they found a conference obscure enough that there was no referee present who recognised that it was a poor duplication of existing work and got it published there.

The point here is that the referees aren't just providing comments because they want to exercise their typing skills. If you get a paper back with comments that there's a serious problem with it then there (usually) really is a serious problem with it that needs to be addressed. Even if it's quite obvious that the referees are idiots and don't understand your work, they are (hopefully) representative of your audience as a whole and if the referees don't understand it then it's likely that the audience won't either (although obviously there are exceptions - a paper on the wonders of X.509 PKI submitted to an OSS conference full of SSH and PGP users, or a paper on SSH's key- continuity key management submitted to an X.509 PKI conference, is going to be a tough sell no matter how well it's written).

In general though trying a different set of referees isn't going to change anything because they'll find the problem as well. Sure, you can eventually get around it by finding a venue so obscure that no-one will notice, but if you're doing it for the publication credits are you really going to get much value from being listed in the appendix to the apocrypha of the Proceedings of the Bratislava Philological Society?

How to sell sex

2008-01-24T01:22:00.000-08:00

The Economist: Economists let some light in on the shady market for paid sex.

Steven Levitt (an economics professor at the University of Chicago and co-author of “Freakonomics”) presented preliminary findings from a study conducted with Sudhir Venkatesh, a sociologist at Columbia University. Their research on the economics of street prostitution combines official arrest records with data on 2,200 “tricks” (transactions), collected by Mr Venkatesh in co-operation with sex workers in three Chicago districts.

The results are fascinating. Almost half of the city's arrests for prostitution take place in just 0.3% of its street corners. The industry is concentrated in so few locations because prostitutes and their clients need to be able to find each other.

Prostitutes are more likely to have sex with a police officer than to be arrested by one.

Sex without a condom is the norm.

One controversial finding is that prostitutes do better with pimps—they work fewer hours and are less likely to be arrested by the police or preyed on by gang members.

In many respects, the paid-sex industry is much like any other business. Pricing strategies are familiar from other settings. Despite evidence of a myopic attitude towards risk, there have been plenty of recent examples of that in the finance industry too. Illegality and lack of regulation are likely to heighten public-health risks. The Ecuador study concluded that rigorous policing of street prostitution might limit the spread of STIs by directing sex workers into the safer environs of licensed brothels.

Top 10 Data Mining algorithms

2008-01-23T01:21:00.000-08:00

The top 10 algorithms are: C4.5, k-Means, SVM, Apriori, EM, PageRank, AdaBoost, kNN, Naive Bayes, and CART.

Top Ten Best (and Worst) Communicators of 2007

2008-01-08T09:41:00.000-08:00

Here.
To Mike Huckabee (first in the list): barack will win bro!

Bill rocks!

2008-01-08T09:25:00.001-08:00

For the first time, Bill Gates is far cooler than Steve Jobs! Ironically, he is so during his last day at Microsoft:

Strong Social Networks

2007-12-21T04:33:00.000-08:00

A well-written article (pdf) reports on the work by Oxford folks (previous post). Plus, it says that a "pair of Oxford physicists, Neil Johnson and Sean Gourley, have teamed up with social scientists at the Conflict Analysis Resource Center (CERAC), based in Bogotá"

"When the researchers graphed all the attacks within a given conflict, with the number of attacks plotted against the number killed in each, it produces a fat-tailed exponential curve. And the exponent of the function, which determines the curve’s shape, is nearly always the same. “Terrorism and guerrilla warfare everywhere in the world has a signature of about 2.5,” says Gourley. Plotting the distribution of these events over time produces another, distinctive signature.

At 71, physics professor is a Web star

2007-12-20T03:00:00.000-08:00

Professor Walter H. G. Lewin "delivers his lectures [owc] with the panache of Julia Child bringing French cooking to amateurs and the zany theatricality of YouTube's greatest hits. He is part of a new generation of academic stars who hold forth in cyberspace on their college Web sites and even, without charge, on iTunes U, which went up in May on Apple's iTunes Store.

In his lectures at ocw.mit.edu, Professor Lewin beats a student with cat fur to demonstrate electrostatics. Wearing shorts, sandals with socks and a pith helmet — nerd safari garb — he fires a cannon loaded with a golf ball at a stuffed monkey wearing a bulletproof vest to demonstrate the trajectories of objects in free fall.

He rides a fire-extinguisher-propelled tricycle across his classroom to show how a rocket lifts off." Source

The stars this week included Hubert Dreyfus, a philosophy professor at the University of California, Berkeley, and Leonard Susskind, a professor of quantum mechanics at Stanford.

Last week, Yale put some of its most popular undergraduate courses and professors online free. The list includes Controversies in Astrophysics with Charles Bailyn, Modern Poetry with Langdon Hammer and Introduction to the Old Testament with Christine Hayes.

The weakness of weak ties

2007-12-19T06:07:00.000-08:00

This is a very interesting paper: Structure and tie strengths in mobile communication networks. It studies the communication patterns of millions of mobile phone users by arranging them in a big weighted social network. The weight between two individuals corresponds to the aggregated duration of calls between them.

Findings:
"Weak ties appear to be crucial for maintaining the network’s structural integrity, but strong ties play an important role in maintaining local communities. Both weak and strong ties are ineffective, however, when it comes to information transfer, given that most news in the real simulations reaches an individual for the first time through ties of intermediate strength." ..."The speed of spread then depended on the strength of each link. The results suggest that information spreads most quickly via links of intermediate strength, or medium length calls. This is because information spreads slowly through weaker links, or shorter calls, and stronger links tend to bind only a limited number of people."

Consequence:
To enhance the spreading of information, one needs to intentionally force it through the intermediate- to weak-strenght ties (while avoiding hubs!)

You, poor? Out!

2007-12-18T08:53:00.000-08:00

Switzerland welcomes rich people with its banks and deports poor ones with this new initiative:

Switzerland has started a television campaign in African countries to keep potential illegal migrants from trying to immigrate to Switzerland.

Robustness of community structure in networks

2007-12-17T21:47:00.000-08:00

"The discovery of community structure is a common challenge in the analysis of network data. Many methods have been proposed for finding community structure, but few have been proposed for determining whether the structure found is statistically significant or whether, conversely, it could have arisen purely as a result of chance. In this paper we show that the significance of community structure can be effectively quantified by measuring its robustness to small perturbations in network structure. We propose a suitable method for perturbing networks and a measure of the resulting change in community structure and use them to assess the significance of community structure in a variety of networks, both real and computer generated." Source

Does your data follow a power-law distribution?

2007-12-17T21:42:00.000-08:00

Power-law distributions in empirical data (htm)

Science paper (pdf)

How brain catalogs info. PageRank-style?

2007-12-17T00:38:00.000-08:00

"Human Brain Cloud: a multiplayer word association game that started with a single word ("volcano") and has since taken on a life of its own. Players are given a word, which is culled from the database of previously entered words, and asked to enter the first thing that comes to mind. As people interact with the game it collects data about word associations that can be formed into a giant network (the cloud)."

"Researchers at the University of California recently conducted a study in which they found evidence to suggest that our brains catalog and rate the relevance of information by forming connections between data. The researchers compared the brain's system to Google's PageRank algorithm"Source

" The investigators found that a word’s “PageRank” was a good predictor of how often it would show up when people were asked to think of words that start with A, with B, and so on.

When it came predicting these results, “PageRank” beat two other seemingly reasonable ranking systems: tallies of how often words show up in ordinary writing; and a simple count of direct “links” to a word that doesn’t consider how many words, in turn, link to those linking words.

In the PageRank formula, a page gains “importance” based on how many other pages link to it. But links from pages that are themselves “important,” confer more importance than those that aren’t. Thus, importance can be thought of as flowing through the Web’s link network toward the most highly “linked-in” sites.

One explanation for the new findings, wrote Griffiths and colleagues, could be that connections among brain cells work similarly to Web links. Cells that are targets of many connections might become more active than others, in the same way that highly linked-in websites are deemed more important." Source

Recommenders Everywhere - WikiLens

2007-12-16T23:34:00.000-08:00

Here is the talk. "Suppose you have a passion for items of a certain type, and you wish to start a recommender system around those items. You want a system like Amazon or Epinions, but for cookie recipes, local theater, or microbrew beer. How can you set up your recommender system without assembling complicated algorithms, large software infrastructure, a large community of contributors, or even a full catalog of items?

WikiLens is open source software that enables anyone, anywhere to start a community-maintained recommender around any type of item. We introduce five principles for community-maintained recommenders that address the two
key issues: (1) community contribution of items and associated information; and (2) finding items of interest. Since all recommender communities start small, we look at feasibility and utility in the small world, one with few users, few items, few ratings. We describe the features of WikiLens, which are based on our principles, and give lessons learned from two years of experience running
wikilens.org."

A mobile phone that can buy clothes for you

2007-12-16T23:23:00.000-08:00

"A revolutionary new mobile phone will soon be able to let shoppers snatch a photo of clothes they want before ordering them online. Nokia is currently developing the device which lets you buy clothes, furniture or holidays in the High Street — without going into a store. Buyers can now avoid queuing at the checkout and even buy clothes simlpy by looking through a shop window and taking a photo of the window display. The phone then uses image recognition software to find the same object on the Internet." Source.

What's on CIO wish lists?

2007-12-10T00:38:00.000-08:00

Number 5: Collaboration Technologies "Web 2.0 and social networking may be becoming candidates for the mainstream, although some CIOs have their reservations. Bob Worrall, for example, CIO of Sun Microsystems, reckons to have talked to well over 100 of his contemporaries over the past year and believes that social networking represents a new threat. There is a lot of information out there on blogs and wiki, but there is no easy way to harvest that information and make it available to the organisation” he says. Sun, however, has created a virtual Californian building in cyberspace and is experimenting with its use as a meeting place for remote staff... RM, the supplier of IT to UK schools, places collaboration and mobility at the top of the list... One of the biggest challenges is to evaluate Web 2.0 opportunities and select those which will add real value to the business" FT

Eight business technology trends to watch (McKinsey)

The Natural Pattern Behind our Votes

2007-12-10T00:19:00.000-08:00

From 30 years of elections around the world: "The most important factor determining a candidate’s success compared with his rivals in the same party turns out to be his or her personal ability to connect with the public."

How opinions form?
Person-to-person process is enough to explain the data! "In their model, they supposed that each candidate starts out trying to convince others to vote in their favour. Those he or she convinces, then try to convince others. These influences percolate through the scoial net until everyone has made a decision."
Consequence
Candidates should focus on WHO they contact - influential people may easily convince others.
More on this pdf.

Lightweight Distributed Trust Propagation

2007-12-09T15:30:00.001-08:00

I just finished to present our work at ICDM. Here are the slides (also in ppt).

| View | Upload your own

Trust Bootstrapping: TRULLO @ Mobiquitous

2007-12-09T15:25:00.000-08:00

At Mobiquitous 2007, we presented TRULLO. Here are the slides (some animations do not work properly in slideshare, sorry). A brief description follows.

Situation: Using mobile devices, such as smart phones, people may create and distribute different types of digital content (e.g., photos, videos). One of the problems is that digital content, being easy to create and replicate, may likely swamp users rather than informing them. To avoid that, users may run trust models on their mobile devices. A trust model is a piece of software that keeps track of who provides quality content and who does not.

Problem: Devices should be able to set their initial trust for other devices. One way of doing so is for devices to learn from their own past experiences. To see how, consider the following quotes about human trust: ``We may initially trust or not trust those involved on our projects based on past experience'', and ``If your boyfriend is unfaithful, you won't initially trust the next man you date'' :-) Algorithms that model human trust on pervasive devices, one might say, ought to do the same thing - they should assign their initial trust upon `similar' past experiences.

Existing Solutions: Existing solutions usually require an ontology upon which they decide which past experiences are similar, and, in so doing, they require both that the same ontology is shared by all users (which is hardly the case in reality) and that users agree on that ontology for good (ie, the ontology is not supposed to change over time) :-(

Proposal: TRULLO gathers ratings of past experiences in a matrix, learns staticial "features" from that matrix, and combines those features to set initial trust values. It works quite well in a simulated antique market and its implementation is reasonably fast on a Nokia mobile phone.

Future: TRULLO does not work if one does not have past experiences. That is why we will propose a distributed trust propadation algorithm (pdf)

Here comes another bubble

2007-12-07T05:12:00.000-08:00

The wireless epidemic

2007-11-26T22:45:00.000-08:00

The wireless epidemic by Jon Kleinberg

At one end are network models that reflect strong spatial effects, with nodes at fixed positions in two dimensions, each connected to a small number of other nodes a short distance away [9]. At the other end are ‘scale-free’ networks, which are essentially unconstrained by physical proximity, and in which the number of contacts per node are widely spread [14]. Models based on human travel data occupy an intermediate position in this spectrum of spatial constraints. The different network structures lead in turn to qualitative differences in the way epidemics spread: whereas epidemics can persist at arbitrarily low levels of virulence in scale-free networks[14,15], epidemics in simple two-dimensional models need a minimum level of virulence to prevent
them from dying out quickly [9].

Bluetooth ...is disrupting this dichotomy by making possible computer-virus outbreaks whose progress closely tracks human mobility patterns. These types of wireless worm are designed to infect mobile devices such as cell phones, and then to continuously scan for other devices within a few tens of metres or less, looking for new targets. A computer virus thus becomes something you catch not necessarily from a compromised computer halfway around the world, but possibly from the person sitting next to you on a bus, or at a nearby table in a restaurant.

9. Durrett, R. SIAM Rev. 41, 677–718 (1999).
14. Pastor-Satorras, R. & Vespignani, A. Phys. Rev. Lett. 86, 3200–3203 (2000).
15. Berger, N., Borgs, C., Chayes, J. T. & Saberi, A. I. Proc. 16th ACM Symp. Discr. Algor. 301–310 (ACM, New York, 2005).

The impact of social structure on economic outcomes

2007-11-26T10:08:00.001-08:00

Some extracts from The impact of social structure on economic outcomes.

4 core principles:

1) Norms and Network Density. ... the denser a network, the more unique paths along which information, ideas and influence can travel between any two nodes. Thus, greater density makes ideas about proper behavior more likely to be encountered repeatedly, discussed and fixed; it also renders deviance from resulting norms harder to hide and, thus, more likely to be punished. ... larger groups will have lower network density because people have cognitive, emotional, spatial and temporal limits on how many social ties they can sustain.

2) The Strength of Weak Ties. More novel information flows to individuals through weak than through strong ties. Because our close friends tend to move in the same circles that we do, the information they receive overlaps considerably with what we already know. ...This is so even though close friends may be more interested than acquaintances in helping us; social structure can dominate motivation. This is one aspect of what I have called “the strength of weak ties” (Granovetter, 1973, 1983). ... if cliques are connected to one another, it is mainly by weak ties. This implies that such ties determine the extent of information diffusion in large-scale social structures. One outcome is that in scientific fields, new information and ideas are more efficiently diffused through weak ties.

3) The Importance of “Structural Holes.” Burt (1992) extended and reformulated the “weak ties” argument by emphasizing that ... the strategic advantage that may be enjoyed by individuals with ties into multiple networks that are largely separated from one another. Insofar as they constitute the only route through which information or other resources may flow from one network sector to another, they can be said to exploit “structural holes” in the network. ... One reason resources may be unconnected is that they reside in separated networks of individuals or transactions. Thus, the actor who sits astride structural holes in networks (as described in Burt, 1992) is well placed to innovate.

Prospective employers and employees prefer to learn about one another from personal sources whose information they trust. This is an example of what has been called “social capital” (Lin, 2001). ... for goods where assessment is difficult, such as used cars, legal advice and home repairs, one-quarter to one-half of purchases in the United States are made through personal networks.

Studies of peasant markets often suggest that “clientelization,” defined as dealing exclusively with known buyers and sellers, raises prices above their competitive level

Social relations are also closely linked to productivity. Economic models attribute productivity to personal traits, modifiable by learning. But one’s position in a social group can also be a central influence on productivity, for several reasons. One is that many tasks cannot be accomplished without serious cooperation from
others; another is that many tasks are too complex and subtle to be done “by the book” (which is why the “rulebook slowdown” is a potent labor weapon) and require the exercise of “tacit knowledge” appropriable only through interaction with knowledgeable others.

“loyalty systems”—attempts to elicit cooperation from workers deriving not only from incentives but also from identification with the firm or with some set of individuals that encourages high standards and productivity.

Your personal data? If you can't take it out, don't put it in

2007-11-26T09:34:00.000-08:00

Many companies are going open. Are they? Take OpenSocial (APIs by Google), Open Handset Alliance (alliance of 34 companies led by Google), and Open Media (an initiative announced by Bebo). What is common to all three initiatives, apart from the use of the word "open", is that none is directly aimed at benefiting the user (here).

Google is clearly "not responding to consumer needs. The applications it has
demonstrated using Android are readily available on existing phones and operating systems. Users are not crying out for yet another interface for their phones".

Tim O'Reilly: "We don't want to have the same application on multiple social networks, we want applications that can use data from multiple social networks".

In another FT's article: "the technology industry has little financial incentive to reduce switching costs. While users are free to switch from one service to another at any time, the critical question is: can they take their data with them? Can they take their photos, their videos, their e-mails? And how easy is that? Data are often stored in proprietary file formats, which are protected by patents, and those are controlled by software and service vendors."

"Which raises the question: Do you actually own your own data?" The answer is unfortunately a qualified no! A very interesting research direction, right?

Cold Reading, Statistical Discrimination and Initial Trust

2007-11-25T21:42:00.000-08:00

"Cold reading is a technique used to convince another person that the reader knows much more about a subject than they actually do. Even without prior knowledge of a person, a practiced cold reader can still quickly obtain a great deal of information about the subject by carefully analyzing the person's body language, clothing or fashion, hairstyle, gender, sexual orientation, religion, race orethnicity, level of education, manner of speech, place of origin, etc. This technique is also called offender profiling. Cold readers commonly employ high probability guesses about the subject, quickly picking up on signals from their subjects as to whether their guesses are in the right direction or not, and then emphasizing and reinforcing any chance connections the subjects acknowledge while quickly moving on from missed guesses".

This definition of cold reading reminded me of Posner's (harsh) review of Blink - Blinkered (html). There are two points from Posner's essay that may suggest how person A may set her initial trust in B. The first point may suggest that A does so based not only on B's behavior but also on the (social) group(s) to which B belongs. The second point may remind us that: asking for recomendations about B may be costly; and that Bayesian reasoning may help in rationally deciding whether to trust. Here are the two (by now-coveted) points:

(1) "If two groups happen to differ on average, even though there is considerable overlap between the groups, it may be sensible to ascribe the group's average characteristics to each member of the group, even though one knows that many members deviate from the average. An individual's characteristics may be difficult to determine in a brief encounter, and a salesman cannot afford to waste his time in a protracted one, and so he may quote a high price to every black shopper even though he knows that some blacks are just as shrewd and experienced car shoppers as the average white, or more so. Economists use the term "statistical discrimination" to describe this behavior. It is a better label than stereotyping for what is going on in the auto-dealer case, because it is more precise and lacks the distracting negative connotation of stereotype, defined by Gladwell as "a rigid and unyielding system." But is it? Think of how stereotypes of professional women, Asians, and homosexuals have changed in recent years. Statistical discrimination erodes as the average characteristics of different groups converge."

(2) " Such pratfalls, together with the inaptness of the stories that constitute the entirety of the book, make me wonder how far Gladwell has actually delved into the literatures that bear on his subject, which is not a new one. These include a philosophical literature illustrated by the work of Michael Polanyi on tacit knowledge and on "know how" versus "know that"; a psychological literature on cognitive capabilities and distortions; a literature in both philosophy and psychology that explores the cognitive role of the emotions; a literature in evolutionary biology that relates some of these distortions to conditions in the "ancestral environment" (the environment in which the human brain reached approximately its current level of development); a psychiatric litetature on autism and other cognitive disturbances; an economic literature on the costs of acquiring and absorbing information; a literature at the intersection of philosophy, statistics, and economics that explores the rationality of basing decisions on subjective estimates of probability (Bayes's Theorem); and a literature in neuroscience that relates cognitive and emotional states to specific parts of and neuronal activities in the brain. "

Is Britney Spears Spam?

2007-11-23T00:17:00.000-08:00

From my old blog (21st August 2007):
In the last post, I was raving on about ..., uhm, probably about trust bootstrapping, right? :-) I went from a definition of cold reading to a very personal interpretation of Posner's review of Blink. Now, in the same vein (i.e., keeping on being delirious), I move on this nice paper, which carries out (in a way) not offender profiling but MySpace user profiling.
Title: Is Britney Spears Spam? (pdf)

Problem: In social network websites (e.g., MySpace), to decide whether to accept invitations to connect, users manually examine the senders' profiles. However that may be time consuming!

Existing Solutions: One may automate the acceptance of invitations by having users running trust propagation algorithms.

Complication: The authors write that using current trust propagation algorithms may be less than desirable since trust both decays with the number of hops and is usually one-dimensional.

Proposal: Use machine learning techniques to classify user profiles. The classification describes a profile across two dimensions: sociability and promotin. Based on these dimensions' values for a profile, users then decide whether
to accept the invitation of that profile's user. To come up with a dataset on which to evaluate their algorithm, the authors randomly select and rate by hand MySpace users.
Future: I would:
> Apply a new trust propagation algorithm (pdf) to avoid trust decay and apply TRULLO (pdf) to handle multi-dimensional trust.
> Look at literature on criminal profiling. In UCL's main library, I noticed many books about criminal profiling. I wonder whether those books could inform a (future) paper titled "On profiling (not only criminals but) Web 2.0 users" ;-)
> Look at literature on statistical discrimination (previous post) and on customer profiling (mining customer data).
> Consider Tim Finin's comments:"It would be interesting to see how well various measures of the network structure around false and true profies serve as features. I think this is very similar to the problem of recognizing spam blogs (splogs). In our work, we’ve found that local features work well, but splogs can also be recognized by looking at the network structure as well."

Efficient and Decentralized PageRank Approximation

2007-11-22T23:44:00.000-08:00

I read this very well-written paper. The authors set out to design a way to compute (an approximated) pagerank in a distributed and efficient way.

"Starting with the local graph G of a peer, the peer first extends G by adding a special node W, called world node since its role is to represent all pages in the network that do not belong to G. An initial JXP score for local pages and the world node is obtained by running the PR algorithm in the extended local graph G' = G+W.
...
we take all the links from local pages to external pages and make them point to the world node. ... as the peer learns about external links that point to one of the local pages, we assign these links to the world node."

Optimized Merging

"At a peer meeting, instead of merging the graphs and world nodes, we could simply add relevant information received from the other peer into the local world node, and perform the PR computation on the extended local graph and still the JXP scores converge to the global PR scores."

Fortune companies don't blog

2007-11-22T10:47:00.001-08:00

"Just 6 percent of the Fortune 500 companies have one [blog], according to Socialtext's Fortune 500 Blogging Wiki.

Why is this? If blogging is still not widespread among companies, it is basically because they are afraid to lose control of their messages , they fear the transparency effect and are not altogether convinced of the legal limits of this medium."

http://www.iese.edu/en/files/6_34173.pdf

How innovation happens in Silicon Valley

2007-11-19T02:48:00.000-08:00

NESTA - How innovation happens in Silicon Valley, London 20/11/07
George Osborne MP will deliver a keynote address. Reid Hoffman, Megan Smith and Javes Slavet will take part in an interctive panel discussion to consider why the Silicon Valley has been so successful, and what lessons can be learned for the UK.

The death of mass advertising?

2007-11-18T00:09:00.001-08:00

Facebook Tries 'Social Advertising'. ..."a Facebook user who rents a movie on Blockbuster.com will be asked if he would like to have his movie choice broadcast out to all his friends on Facebook. And those friends would have no choice but to receive that movie message, along with an ad from Blockbuster."

MySpace reveals 'targeted' ads - "a pilot scheme that allows it to sell advertisements targeted to the individual tastes and interests of its millions of users...[It] will give advertisers the ability to drill down into 100 different user segments. This will allow them to differentiate between fans of romantic comedy films and action films, for example."