Wednesday, July 25, 2012

Simon Urbanek Rserve and FastRWeb

In a session from Simon Urbanek of ATT labs, discussing "Distributed R and the cloud"... more shareable, parallizable, distributed, flexible ways of using R.

Rserve:

* initialize R
    - load data, code, packages
* Listen for incoming connections
* Fork on accept... multiple client(s)' processes at same time (if multiple processers)
* Communication:
    -TCP/IP
    - Unix sockets
    - QAP binary protocol
    - HTTP/HTTPS
    - WebSockets - persistent browser / R connection
    - allows switching of types...
* Rserve security
    - authentication, encryption, user separation

Uses of Rserve (goals/features):

* FastRWeb
* Rserve.cluster - as backend for snow/parallel... no more need for ssh
* Rserve + RCassandra - move computation to the data
* R in the cloud
    - R console in the cloud
    - code/notebook sharing, reporting, interactive graphics

More on FastRWeb

* Maps URLs to script execution
http://foo/cgi-bin/R/bar?n=100  becomes  source("../bar.R")
* Requires CGI or PHP or all-in-one Rserve solution
* Very fast response, parallel access
* Automatically supports most common tasks
* Very easy to build web-based tasks

Rserve.cluster

* Easy to get started
* Uses SNOW
* pre-loading of data, packages, code
* shared initial state (don't have to push data, etc.)

Rserve and the Cloud

* The Cloud: R's in-memory model is "in" again
* Faster graphics, WebGL, etc.
* Interactive Web Graphics
* Large data in the browser
* The wplot command looks really neat

Integration with knitr:

* Integration with Rmarkdown (via knitr, markdown) for R-web-notebook functionality

Streaming:

* no R API for streaming (all ad-hoc right now) - Need something to unify/standardize this
* Asynchronous evaluation: background
   - Allows trigger functions associated with external events


FastRWeb links:


R-forge page:
http://www.rforge.net/FastRWeb/

Package:
http://cran.r-project.org/web/packages/FastRWeb/index.html

Package Index:
http://www.rforge.net/doc/packages/FastRWeb/html/00Index.html

R-blogger:
http://www.r-bloggers.com/tag/fastrweb/

A setup blog page from another blogger:
http://jayemerson.blogspot.com/2011/10/setting-up-fastrwebrserve-on-ubuntu.html