Last week the .NET Twitterverse erupted when a developer from Harrisburg, PA named Justin Rusbatch released Compilify. Compilify is a simple web-based utility that allows you to write C# and then run it from your web browser. Justin deployed the site on AppHarbor so we reached out to him to learn more about it.

Tell us a little bit about Compilify...

JR: Compilify (pronounced “compile-ify”) is site for people can experiment with, collaborate on, and share C# code. Any errors will be displayed as you type and with the click of a button you can compile and execute your code. Each snippet can be saved and shared with others via a unique URL.

Compilify makes the .NET compiler completely portable and accessible through a fast, simple interface that fosters sharing and collaboration. It’s not an IDE in your browser, nor will it ever be. It’s simpler than that. You shouldn’t need to launch an IDE or create a new console project to experiment with a new idea that only takes a few lines of code. A developer’s time is incredibly valuable. Spending too much time thinking about a problem without actually trying solutions results in over-engineered solutions and -- if left untreated -- can lead to analysis paralysis.

Compilify is the tool you will reach for when you don’t have time to deal with a full IDE!

It also has a lot of potential as an educational tool for people who don’t know C# but want to try it. Downloading, installing, and launching Visual Studio can intimidate new developers. Some developers might not install Visual Studio because of all the other applications that it install with it. Compilify lets users get their hands dirty without needing to install anything. Not even a browser plugin.

As I was watching the tweets about Compilify on Thursday, there was one in particular that jumped out and grabbed my attention:

“just wrote and compiled C# from my droid!” -- @SpoonTheGreater

I’ll pause for a moment so you can reflect on how incredibly awesome that is.

Have you calmed down a little bit? Good, prepare to get excited again. Compilify lets you write, compile, and execute C#, quickly and easily, from any platform with a web browser without installing anything!

What inspired you to build Compilify?

JR: Compilify was inspired by many things. One of the primary sources of inspiration was the C# Interactive window that the Roslyn CTP brings to Visual Studio. This window provides a REPL environment that can be used to execute statements against the projects they’re developing and receive immediate results. The driving goal behind Compilify is to bring that same interactive environment to the web.

What technologies are you using?

JR: The website itself is an ASP.NET MVC4 project.

When I started the project, I decided to use Redis for queueing code execution and messaging between the web application and the workers. Initially I was going to use it as the primary data store for the application, but after a few days I decided to switch to another NoSQL solution that would be supplemented by Redis. I decided to go with MongoDB for this because of its availability as an add-on service through AppHarbor.

SignalR is currently being used as a message relay. After a worker finishes executing a user’s code, the result is published to a Redis pub/sub channel that the web application is subscribed to. SignalR is then used to "push" that result to the client that originally sent the code. I have plans to implement features that make more use of SignalR in the future.

Which tools or frameworks did you like most?

JR: Booksleeve, the Redis client I chose. Up-to-date documentation was a little sparse, but the API was straightforward once I figured out the basics. I love simple and straightforward libraries.

What was something unexpected you learned while building it?

JR: The whole project was an awesome learning experience. I’ve been excited to play with Roslyn since I attended a session on the future of C# and VB by Anders Hejlsberg at Build conference last September. But, until Compilify, I didn’t have a good use case for it aside from the tutorials that come with it.

Protecting myself from the code submitted by users was the most challenging part of the project. Developers are extremely clever and creative, so offering to execute their code for them is a little risky. A majority of my time working on the site was spent making the site as secure and stable as I could, without making the users feel like they didn’t have enough permissions for the site to be useful. I think I’ve reached a good balance at this point. A huge resource for me while working on securing the application was a blog post by Filip Ekberg describing his efforts at building an IRC bot based on Roslyn.

It’s my understanding that the Roslyn team is working to make this easier in future releases.

What advice would you give to other developers running their apps on AppHarbor?

JR: Take advantage of the add-ons. There are a lot of great tools available as add-ons for little or no cost that can be integrated seamlessly through AppHarbor.

What are your plans for Compilify in the future?

JR: I’m really excited about Compilify’s future. There’s still a lot of unrealized potential for Roslyn My to-do list includes (in no particular order):

• Allow class declarations (this might even be implemented by the time this is posted).
• Provide the ability to import/export snippets to/from other popular pastebins.
• Provide more details about errors in the code, without being distracting.
• Add more features to facilitate collaboration, including the ability for multiple users to edit the same snippet simultaneously.
• Results as you type, like validation errors are displayed now.
• Possibly add support VB and F#. Maybe. Frankly, I’m always writing in C#, so it really depends on how many users want support for the other languages.

I’m also eager to hear suggestions from users. This is an open source project, so feel free to submit enhancements via the issues list or pull requests!

Where can people contribute to the project?

JR: The source code for Compilify.net is available on GitHub under the MIT license.

Where can people find you on the internet?

JR: I’m on Twitter and GitHub

Last month we launched the Background Workers beta. If you're unfamiliar with background workers, they're very easy to get started with. Any .exe files included in your project's build output will be run at each deploy. As one of the most frequently-requested features, we've already seen many of our customers take advantage of this new feature.

Background Workers are simple to write and deploy because there's nothing special about them; they're just regular .NET console apps. This gives you a lot of flexibility in the types of workers you can write. You can do one-time processing, scheduled tasks or handle long-running tasks like processing items from a queue.

In this post we'll cover some of the basics of writing your first background worker.

One-time processing

Many apps require tasks to be run the first time they are run. For instance you might need to populate a database with fresh data in a staging environment or update a CDN with the latest version of your site's static assets. To run a one-time task like this with an AppHarbor Background Worker, add a new Console project to your solution. Here's a basic example of a worker that's not very useful:

namespace WorkerDemo
{
    class Program
    {
        static void Main(string[] args)
        {
            Console.WriteLine("AppHarbor background workers rock!");
            // do something useful :)
        }
    }
}

The next time you deploy your app your worker will be executed. If it completes without an error, it won't be run again until your next deploy. If there's an error during execution, the worker will be restarted.

Error Handling and Capturing Output

Current we do not provide logs of the console output generated from background workers. We're working on adding this feature in the future, but for now you'll need to log your own errors. The best way to do this is to wrap your entire worker in a try/catch block and grab the exception details to store or notify someone that something went wrong. Below, we'll see how this could be accomplished using NLog and the Logentries add-on.

Capturing exceptions is useful, but we can take things a step further and capture the output of all Console.Write and Console.WriteLine calls. We've written a helper class that captures the console output written by those two methods and stores it for you to access later. You can find the code for the ConsoleMirror utility on our GitHub page. As a bonus, our ConsoleMirror class not only captures the output, but still sends it to the console so that when you run your worker locally you can still see what's happening.

To get started, install the ConsoleMirror NuGet package to add it to your project. To wire it up, add the following lines to your app:

using AppHarbor;
namespace WorkerDemo
{
    class Program
    {
        static void Main(string[] args)
        {
            ConsoleMirror.Initialize();

            try
            {
                Console.WriteLine("AppHarbor background workers rock!");
            }
            catch (Exception ex)
            {
                Console.WriteLine(ex.Message);
                Console.WriteLine(ex.StackTrace);
            }

            var output = ConsoleMirror.Captured;
            // do something useful with the output
        }
    }
}

Doing Something Useful with the Captured Output

Now that we have a copy of the output from our worker we could email it (using the Sendgrid or Mailgun add-ons, of course) or log it. Using the Logentries add-on with NLog, we can store the console output like this:

using System;
using AppHarbor;
using NLog;

namespace WorkerDemo
{
    class Program
    {
        private static Logger logger = LogManager.GetCurrentClassLogger();
        static void Main(string[] args)
        {
            ConsoleMirror.Initialize();

            try
            {
                Console.WriteLine("AppHarbor background workers rock!");
            logger.Info(ConsoleMirror.Captured);
            }
            catch (Exception ex)
            {
                Console.WriteLine(ex.Message);
                Console.WriteLine(ex.StackTrace);
                logger.Error(ConsoleMirror.Captured);
            }
        }
    }
}

The Logentries web interface will now include a history of what our worker produced while running.

More on Background Workers

In a future posts we'll cover how to write long-running workers and how to use background workers to run tasks on a schedule. In the meantime, we'd like to see what you've done with background workers in your apps. Send us an example we can include on our blog, or write about it on your own blog and we'll send some sweet AppHarbor swag your way.

You can now provision Searchify full-text search as an add-on on AppHarbor. We're really happy to partner with Searchify, especially because the Searchify service is a drop-in replacement for the IndexTank add-on that we had to retire after IndexTank was acquired by LinkedIn last year.

Searchify provides an incredibly fast full-text search service with support for advanced features such as facets, snippets and autocomplete.

To get you started quickly, we've created a sample application that loads the major works of Shakespeare (courtesy of project Gutenberg) into Searchify to let you quickly figure out which one contains the term "Capulet". You can play around with the sample app here.

The sample application uses the IndexTankDotNet client library by Lee Dumond. There seems to a slight problem with IndexTankDotNet and the latest version of RestSharp but a pull request with a fix is under review.

UPDATE: Lee has merged the pull request and updated the NuGet package.

Add full-text search to your AppHarbor-hosted app right now by provisioning the Searchify add-on.

You asked for it, and we teamed up with New Relic to build it: New Relic is in the AppHarbor add-on catalog. You can now seamlessly provision New Relic's superb performance monitoring solution to your AppHarbor-hosted applications and have the necessary configuration automatically injected into your application. Currently only the free Standard plan is available, but we expect the paid Profesional plan to follow shortly.

Monitoring your application with New Relic (optionally combined with load-generation from Blitz.io add-on) is a great way to determine whether you need to start scaling your app. It's also a great way to record and keep track application errors if your needs exceed the basic AppHarbor application error tracking.

New Relic's .NET Monitoring Agent is installed on all AppHarbor application servers. If you don't want to take advantage of the add-on, you can add your own configuration and license key to your app and push that to AppHarbor. The agent will pick this up and start monitoring your app.

AppHarbor users who provision New Relic get a 15 day free trial of the professional plan courtesy of the great guys at New Relic.

We're excited to announce that you can now provision the IronMQ message queuing add-on to apps running on AppHarbor. IronMQ is built and run by the guys at Iron.io and they have lots of experiencing hosting great add-ons.

Iron.io has built a .NET wrapper for the IronMQ API, making using the service from .NET apps incredibly easy. Check out the AppHarbor knowledge base for info on how to configure the wrapper. Finally, we've created a simple application that uses IronMQ to pass tasks and results between an ASP.NET MVC frontend and an AppHarbor backgroundworker backend. You can use the same pattern in your AppHarbor-hosted applications to pass asynchronous tasks (like sending emails) to backgrounder-worker processes.

We're excited to be partnering with Iron.io and we think you'll find IronMQ to be a great addition to the AppHarbor add-on catalog.

UPDATE: Announcement on Iron.io blog

We've just added the CloudAMQP queue service to the AppHarbor add-on catalog. CloudAMQP is based on RabbitMQ, a mature message queueing system with good .NET client support. The CloudAMPQ guys have written a great guide on how to use the service from AppHarbor hosted applications.

Using queue services like CloudAMQP is a good way to coordinate tasks between AppHarbor web- and background-workers. Instead of completing time-consuming jobs (such as sending emails) synchronously while processing a request, you can enqueue the job, return a request response to your user and have a background worker pick up the message and asynchronously complete the job. Using this pattern will help you build more responsive and robust applications on AppHarbor. We expect to soon have a tutorial and screencast on the topic ready.

We're excited to expand the AppHarbor offering with a solid queue service and we look forward to your feedback on the CloudAMQP add-on!

Today we're launching a greatly extended AppHarbor API. The API lets you easily write apps and scripts that create, manipulate and monitor applications on AppHarbor. You can also use the API to integrate application management (such as dynamically adding hostnames) directly into your AppHarbor hosted applications.

The API uses the same URIs you currently use when navigating the appharbor.com site. Data is sent and received with JSON. Authorization is done using OAuth and you need to register an API-client on AppHarbor to get started. There are additional details on the API Overview page. We're launching with support for the following resources:

• Applications
• Collaborators
• Configuration Variables
• Hostnames
• Service Hooks
• Builds
• Tests
• Errors
• Users

The API is in beta and creating and modifying the currently supported resources will not impact your monthly bill (except for hostnames, so watch out when adding those). We'd love to hear any and all feedback you might have on the API. Also note that we might make changes to the API while it's still in beta, but we'll try to keep users abreast of any developments. We're also working on extending the API to include more of the paid stuff so that you can do neat tricks like write scripts that automatically scale your application based on load.

Our experience releasing just tiny bits of API functionality (mainly Build URLs and Service hooks) is that the AppHarbor community has jumped at the chance to create great utilities like TweetHarbor (by Seth Webster) and MaintMan (by Drew Miller). We can't wait to see what you come up with this much expanded API!

This screencast walks you through creating an application suitable for deployement to AppHarbor background workers. The application fetches the newest tweets about "appharbor" with LinqToTwitter and stores the tweets in RavenDB using the RavenHQ add-on.

The screencast also shows how to avoid committing binaries to source control by downloading dependencies using NuGet during build and demonstrates setting up a git repository up for deploying to AppHarbor.

Resources:

• The sample source code on GitHub
• Background workers announcement
• Knowledge base article about background workers
• Use NuGet Package Restore to avoid pushing packages to AppHarbor
• RavenHQ add-on launch post
• RavenDB server download (for running the RavenDB server locally)
• RavenDB website
• Tekpub's RavenDB production
• LINQ to Twitter project site

They're here! AppHarbor background workers are now in beta. To start running background tasks right now, go to your application's subscription page and add at least one background worker. The next time your application is deployed, AppHarbor will identify any .exe files in the build output of your application and start running them. This makes running background tasks on AppHarbor as easy as adding a console application project to your solution. An example console app could look like this:

public static void main(String[] args) 
{
    while(true)
    {
        //Do something
        Thread.Sleep(30000)    
    }
}

When deciding how to support background tasks on AppHarbor, we wanted to take the same approach we used for ASP.NET apps: Make getting started as simple as possible and require as few modifications to your existing code as possible. That's why we settled on running .exe files found in the application build output.

Background workers share their configuration with web workers and can access application add-ons and other resources in the same way that web workers can. For existing AppHarbor applications there are no deployment changes until a background worker is added to the subscription. Once at least one background worker is added, then any .exe files will launched, and both web and background workers will be automatically updated when you push new versions of your code.

AppHarbor background workers are not only useful for running auxiliary tasks associated with web applications. You can use background workers to scale any .NET workload that can be run as an executable on AppHarbor. We're excited that background workers lets us expand the AppHarbor value proposition to whole new classes of .NET applications. We will continue to improve and expand the platform to let you run and scale any application on AppHarbor.

If you need additional control over what code AppHarbor runs in background workers, you can take advantage of the solution file convention to not build projects that you don't need AppHarbor to run. If we find an .exe file with a corresponding .exe.config file that has a appharbor.deploy_as_background_worker appSetting set to false, then that particular executable will be ignored.

Applications run in an AppHarbor background worker are isolated in the same way as ASP.NET applications run by web workers. All executables run by a single background worker compete for the same resource allocation (i.e. having multiple copies of the same executable run by a single background worker will not get you more resources). Note that the AppHarbor Terms Of Service prohibit creating multiple applications with the intent to scale without incurring any costs.

If your executable exits with a non-zero exit code, AppHarbor will restart the application. If the exit code is 0, the application will not be restarted, but it will be started again if a new version of your application is deployed.

We do not currently provide any logging of background worker console output, although that is on our TODO-list. The logging add-ons in the AppHarbor add-on catalog catalog will work for properly instrumented background worker applications.

As always, we can't wait to see how AppHarbor users will use this new feature and we welcome feedback and feature requests. If you encounter problems, then check please refer to the AppHarbor support page. Also check out the knowledge base article on background workers.

UPDATE: A screencast demonstrating how to create and deploy a background worker is now available here.

We're excited that so many developers are using AppHarbor to scale .NET applications on our cloud platform. With great power comes great responsibility however, and as we pointed out when scaling was launched in beta, you have to carefully manage any state that has to be shared between multiple web workers running your app.

In tandem with the original launch, we published a guide and an example app demonstrating how to store sessions using the Memcacher add-on. We have changed the sample from always using memcached (annoying, since that means you have to run memcached on your dev machine) to just using memcached when deployed in Release mode. When running in Debug mode the standard in-proc Session State Provider is used. This should make developing and maintaining applications using memcached for session state storage easier.

Below is the Web.Release.Config transformation that you can use to configure the Memcached Session State Provider when an app is deployed to AppHarbor. You don't have to change anything, AppHarbor will automatically insert all relevant settings. Remember that you can test your config transformations if you need to incorporate additional transformation logic.

<?xml version="1.0"?>

<configuration xmlns:xdt="http://schemas.microsoft.com/XML-Document-Transform">
  <configSections>
    <section name="cacheProvider" type="MemcachedProviders.Cache.CacheProviderSection, MemcachedProviders"
      allowDefinition="MachineToApplication" restartOnExternalChanges="true" xdt:Transform="Insert"/>
    <sectionGroup name="enyim.com" xdt:Transform="Insert">
      <section name="memcached" type="Enyim.Caching.Configuration.MemcachedClientSection, Enyim.Caching"/>
    </sectionGroup>
  </configSections>

  <enyim.com xdt:Transform="Insert" >
    <memcached protocol="Binary">
      <servers>
        <add address="foo" port="bar" />
      </servers>
    </memcached>
  </enyim.com>

  <cacheProvider defaultProvider="MemcachedCacheProvider" xdt:Transform="Insert">
    <providers>
      <add name="MemcachedCacheProvider"
          type="MemcachedProviders.Cache.MemcachedCacheProvider, MemcachedProviders"  defaultExpireTime="2000"/>
    </providers>
  </cacheProvider>
  <system.web>
    <sessionState cookieless="false" regenerateExpiredSessionId="true" mode="Custom"
                  customProvider="MemcachedSessionProvider" xdt:Transform="Insert">
      <providers>
        <add name="MemcachedSessionProvider"
             type="MemcachedProviders.Session.SessionStateProvider,MemcachedProviders"
             dbType="none" />
      </providers>
    </sessionState>
  </system.web>
</configuration>