Simon Urbanek Rserve and FastRWeb

In a session from Simon Urbanek of ATT labs, discussing "Distributed R and the cloud"... more shareable, parallizable, distributed, flexible ways of using R.

Rserve:

* initialize R
    - load data, code, packages
* Listen for incoming connections
* Fork on accept... multiple client(s)' processes at same time (if multiple processers)
* Communication:
    -TCP/IP
    - Unix sockets
    - QAP binary protocol
    - HTTP/HTTPS
    - WebSockets - persistent browser / R connection
    - allows switching of types...
* Rserve security
    - authentication, encryption, user separation

Uses of Rserve (goals/features):

* FastRWeb
* Rserve.cluster - as backend for snow/parallel... no more need for ssh
* Rserve + RCassandra - move computation to the data
* R in the cloud
    - R console in the cloud
    - code/notebook sharing, reporting, interactive graphics

More on FastRWeb

* Maps URLs to script execution
http://foo/cgi-bin/R/bar?n=100  becomes  source("../bar.R")
* Requires CGI or PHP or all-in-one Rserve solution
* Very fast response, parallel access
* Automatically supports most common tasks
* Very easy to build web-based tasks

Rserve.cluster

* Easy to get started
* Uses SNOW
* pre-loading of data, packages, code
* shared initial state (don't have to push data, etc.)

Rserve and the Cloud

* The Cloud: R's in-memory model is "in" again
* Faster graphics, WebGL, etc.
* Interactive Web Graphics
* Large data in the browser
* The wplot command looks really neat

Integration with knitr:

* Integration with Rmarkdown (via knitr, markdown) for R-web-notebook functionality

Streaming:

* no R API for streaming (all ad-hoc right now) - Need something to unify/standardize this
* Asynchronous evaluation: background
   - Allows trigger functions associated with external events

FastRWeb links:

R-forge page:
http://www.rforge.net/FastRWeb/

Package:
http://cran.r-project.org/web/packages/FastRWeb/index.html

Package Index:
http://www.rforge.net/doc/packages/FastRWeb/html/00Index.html

R-blogger:
http://www.r-bloggers.com/tag/fastrweb/

A setup blog page from another blogger:
http://jayemerson.blogspot.com/2011/10/setting-up-fastrwebrserve-on-ubuntu.html

latest phyloseq version now 1.1.18

Please update to the latest version. Most importantly, this fixes a bug arising from a not-backward-compatible update to the igraph-package. It sounds like a needed change for that package, but results in most of the packages that use igraph to fail once the latest igraph version is installed.

This has been fixed for phyloseq. More details on the fix, and complete installation instructions can be found at the phyloseq installation instructions page.

Please feel free to post wiki suggestions for examples, new features, and definitely any issues or bugs! I aim to be very fast on responding to user requests, particularly in the cases if there are functional bugs, since I know this can slow down the already-difficult research everyone is doing.

Bugs, other issues, suggestions, and feature requests can all be posted at the phyloseq issues page. In principle you could also post requests for tutorials and examples on the wiki at this page as well, but an alternative might be to post those requests here on this blog, as an empty wiki page on the phyloseq wiki, or by simply sending me an email with specifics and any example code or other data that might help me turn-out a reproducible example as fast as possible.

Thanks!

joey

Great recipes for setting up R/RStudio in the Amazon Cloud

I originally stated that I hadn't tried this yet. I have tried it, and it works great. Amazon's free 1-year micro instance makes this especially great for small applications. Also a very good tool to learn for big modern datasets.

One of the most useful tools is on the Bioconductor website, describing the use of a special R/Bioconductor Amazon Machine Image (AMI):


http://www.bioconductor.org/help/bioconductor-cloud-ami/

http://www.r-bloggers.com/rstudio-in-the-cloud-for-dummies/

http://www.r-bloggers.com/r-ec2-rstudio-server/