Grid Computing using Javascript

I recently had an idea that’s both really interesting and a bit disturbing. Authors of websites could use clients’ computers as a massive grid computer. If you have a problem that you can solve by using map-reduce, you could do the map step in the client’s web-browser, and let them send back the result. Most clients would never know you’re doing this, and by only using a couple of seconds from their processor time, they also wouldn’t notice.

The problems that you can solve this way have some interesting properties: you must never send any private data to the clients and typically the solution should be expensive to find, but easy to verify.

I’m curious what kind of cool appliciations we can build using this technology. If a major site like Facebook would include a tiny bit of javascript that takes 5 seconds of your processor time to help fight AIDS, it would probably be a tremendous help.

I would be very interested in your thoughts on this matter. Are there more problems that we could solve this way? I’m mainly interested in the technical implications, but there are lots of non-technical issues as well. For example: would you need the client’s permission to do this?

About this entry

You’re currently reading “Grid Computing using Javascript,” an entry on Tupil Code Blog

Published:: Tuesday, September 2nd, 2008 at 19:20

Author:: Chris Eidhof

Category:: Code

Tags:: cloud computing, grid computing, Javascript

@tupil on Twitter

We built a new website. Due to popular demand, we also have a portfolio now: http://tupil.com/iphone.html

Both NRC Handelsblad's and De Volkskrant's new iPhone apps are now available! (See http://is.gd/5FiX3 and http://is.gd/5FiX2)

RT @camathome: leuke download op Bright: Mycon, geef je favoriete contact een icoon op je homescreen http://bit.ly/59qpjB #iphone

@gonny Dat zijn de eerste reviews ;)

We just launched myCon, an app for creating apps. How cool is that? Check it out in the App Store: http://tinyurl.com/myconapp

Paul

September 2nd, 2008
19:39 CEST

You’d have to weigh the load on the client machine with the load on the server accepting the request back. I’m just not sure if anyone would be willing to wait for a page to finish loading though. Maybe with the new version of Mozilla coming up soon this will be more effective.

Chris Eidhof

September 2nd, 2008
19:43 CEST

Yes, you probably would want to start computing only after the page has loaded, so the user can still use the page, and then send back the result using Ajax. And the computation should indeed be more expensive than the impact of the extra request that you on the server, much the same as ordinary map-reduce.

Nils Breunese

September 2nd, 2008
19:47 CEST

I also read some time ago that JavaScript is the new loadbalancing. Serving code (JavaScript) to a lot of people and have them execute it is easy, executing code server side for a lot of people is hard.

Corey O'Connor

September 2nd, 2008
20:50 CEST

This is an interesting idea and one worth exploring. Though I wouldn’t try solving any “hard” problems with such a system yet. Still, a few experiments are worthwhile and could serve to explore the problem space.

I was considering starting with a simple distributed ray tracer. A screen space partitioning distributed ray tracer is a well understood problem. An implementation using AJAX shouldn’t run into too much trouble and there would be plenty of existing implementations to compare the resulting system against.

Jay

September 2nd, 2008
20:53 CEST

I can’t find a link to this (and your googling will likely be as good as mine) but there was a chess site that did something “similar”.

When you went there to play chess the page (I don’t remember if it was java or javascript) would submit the board position and get back the computer’s move from the central server. This move was obtained via a database of moves and wasn’t particular unique in implementation.

But what was unique was that if there was no record of a “winning” move (or at least a good one) based on the current board then they’d have your browser compute N-layers down and decide what was most appropriate.

Then they’d send the state & the move back to their servers to be recorded into the “global” database of moves.

Kord Campbell

September 2nd, 2008
20:55 CEST

Have your JS open an iframe for a page you want to crawl/spider. Use the client computer to extract links, parse text, and send you back the results.

Roberto

September 2nd, 2008
20:56 CEST

I once wrote a proof-of-concept pi calculator, using Javascript on the browser. IIRC, given two random numbers a, b between 0 and 1 the chance of c = sqrt(a^2 + b^2) being greater than 1 is proportional to pi. Each client would generate two random numbers and calculate c and send to the server, which would accumulate the statistics and give an estimate of pi. This was before AJAX, so it was all done with a regular form.

It would be interesting to write a program (in Python or Ruby, I don’t know), which given a function to paralelize would generate a web application that offloaded the calculation to Javascript — just open as many browsers as you want, point to the server and let it run… seems like a fun project to do. :)

Brian

September 3rd, 2008
1:30 CEST

Don’t use ruby.

Nico

September 3rd, 2008
20:18 CEST

For this to be useful you’d have to have enough traffic that it makes more sense to farm out the work rather than have servers just do the work between serving pages.
So unless you have a reliable stream of clients, you might want to have a hybrid where the servers do the work when idle and otherwise pass of some jobs as javascript to be evaluated by the client.
The distributed ray-tracing thing could be fun this way – the more people connecting to see the image, the faster the image gets generated.

September 4th, 2008
0:53 CEST

I wrote a small demo (http://dealmeida.net/2008/09/03/using-ajax-to-parallelize-python-functions), though currently it’s not really running in parallel. But the code converts a Python function to Javascript, and runs an AJAX framework to share the computation with browsers.

Francois

September 4th, 2008
7:00 CEST

Google gears has and HTML5 is supposed to have implementations for background javascript processes.

Google Gears:
http://code.google.com/apis/gears/api_workerpool.html

HTML5
http://dev.w3.org/html5/workers/

I am working a project that involves similar ideas, I was going to use a video game that had incentives for leaving your browser going so that you could essentially donate computer time in exchange for time back later.

Obviously, it’s sort of perverse, as I’m sure that a C++ or assembly implementation would be orders of magnitude faster. If your potential market of users is proportionally larger than your potential assembly/C++ market, then you might win out.

Javascript is getting faster and faster, but it’s still no number crunching language.

I had the exact same thoughts on using NP class calculations so that you could cheaply verify.

Email me if you are interested in hearing more about my project.

Lucas

September 8th, 2008
14:28 CEST

I’ve noticed the same. It could be done more efficiently by using a java applet, and that also allows you to use any jvm-able language.

From a malicious perspective, it would be very easy to insert an iframe that holds an applet on a large website that is vulnerable to html injection (xss not required, an iframe would be enough). Strictly theoretical though, of course.

Allan

September 8th, 2008
16:08 CEST

in some place, it’s a violation of human rights thingie.. (i don’t know law, but i know that this is true)

besides, ever heard of law of diminishing returns? that’s why parallel or synchronized computing is usually done within dedicated lines or clustered areas..

ahot

September 25th, 2008
19:53 CEST

Hi Chris! Looks like we think similar. Look at my note: upside-down-cloud-computing where I described the same Idea :)

Maarten Brinkerink

October 5th, 2008
20:58 CEST

Niet helemaal hetzelfde, maar zijn idee over ‘human computation’ is ook erg interessant: http://www.wired.com/techbiz/it/magazine/15-07/ff_humancomp

cody

February 7th, 2009
3:14 CET

Cool. One question. Say one visitor does a bunch of computations , then the next visitor picks up where the previous one left off. How does that previous work get transferred to the next user?
PS. I have a script on my page that this would be cool for.

Jeiel Schalkwijk

January 7th, 2010
13:47 CET

Perhaps a bit late, but I’ve also had this idea. I think it will be very practical once we have x86 in the browser (Google is working on this) and the projects are obvious: SETI@home, Folding@home, etc…

Google is promoting the browser as the OS. A computer would have a minimal OS, with an always-on browser. And in the browser it would be possible to ‘install’ web applications, with 1 click. Apps get a local datastore and can run x86 code.

I’m sure that installing daemons (services for you Windows folks) would also be possible. Daemons that notify you of new blog posts, comments, emails, last minute travel offers, weather alerts, etc.
And a Folding@home daemon that uses your spare cycles, when not on battery power.

Christian Harms

February 9th, 2010
0:03 CET

I build a number cruncher with JavaScript fpr perfect numbers and it can tiled with webWorker (thread Worker). And the backend for starting this app in each visitors browsers …

Tupil Code Blog

Grid Computing using Javascript

About this entry

18 Comments

Have your say

About

Recently

@tupil on Twitter

Tags