We have a new add-on ready for you: ElephantSQL – cloud-hosted PostgreSQL as-a-service.

PostgreSQL is a stable, mature and high-performance open source RDMS and the guys at ElephantSQL have done a great job of packaging that into an awesome service.

We have updated our PostgreSQL-NHibernate example MVC app to also work with ElephantSQL and there's additional .NET sample code and documentation on the ElephantSQL website.

ElephantSQL has a free plan to get you started, try it out right now!

When we launched application scaling more than a year ago one of the enabling features was that AppHarbor automatically injects a machineKey element in the application web.config unless one is already specified. Today, we're announcing changes to the default machinekey settings injected for applications running on .NET 4.0 and 4.5. The changes will be phased in a week from now, on November 14th. The changes are not breaking, but users may need to log back in to sites using ASP.NET Forms Authentication the first time they visit the site after a new version of the site is deployed.

AppHarbor has to inject the machineKey element because an application is scaled by deploying the app on multiple workers. To make sure that forms authentication cookies generated by one worker are also valid when received by other workers, the keys used to encrypt and validate the cookie have to match across workers. Previously, we would always specify SHA1 for the validation attribute of the machineKey element. We chose this setting because it works with all framework versions. The change we're introducing is to upgrade the default validation algorithm for .NET 4.0 and 4.5 applications to HMACSHA256, which is the default for those framework versions. The upgrade will cause previously issued forms auth cookies (and any data encrypted using the MachineKey helper) to become invalid and users will have to log back in to get a new cookie.

If you are running a .NET 4.0 or 4.5 application and you want to opt out of this change, simply add a machineKey element to your web.config that looks like this:

<machineKey validation="SHA1" />

AppHarbor will still inject validation and decryption keys, but the algorithm attribute will be left alone. If you're already specifying a valid (non-autogenerate) machineKey element, your application will not be affected by this change.

As always, if you don't want the hassle of dealing with Forms Authentication and the lack of control over the cookies issued, we recommend using AppHarbor.Web.Security

This post continues our investigations into the more useful parts of Microsoft's premier ORM, Entity Framework. Previous installments include automatic migrations (including screencast) and loading and querying spatial data using Entity Framework. Today, we'll improve the sample code from the migrations post to be completely convention-based and we will also create a repository implementation to access data.

Building the model

You might remember from the automatic migrations post that we built a setup for simple convention-based migrations that are executed seamlessly as the model is expanded and updated. There is one smell in that sample though: We want database schema to be created for all classes that inherit from an abstract Entity class, but what actually defines what goes in the database are the DbSet property declarations in the Context class:

public class Context : DbContext
{
    public DbSet<User> Users { get; set; }
}

That's bad: To add an entity to our data model we have to both create the class and add a property to the DbContext-derived class. Luckily, Rowan Miller from the Entity Framework Team has an example demonstrating how to dynamically create a code first model. Adapting our OnModelCreating() implementation to use that approach results in this:

protected override void OnModelCreating(DbModelBuilder modelBuilder)
{
    ConfigureModel(modelBuilder);

    Database.SetInitializer(new MigrateDatabaseToLatestVersion<Context, Configuration>());

    base.OnModelCreating(modelBuilder);
}

private void ConfigureModel(DbModelBuilder modelBuilder)
{
    var entityMethod = typeof(DbModelBuilder).GetMethod("Entity");

    var entityTypes = Assembly.GetAssembly(typeof(Entity)).GetTypes()
        .Where(x => x.IsSubclassOf(typeof(Entity)) && !x.IsAbstract);
    foreach (var type in entityTypes)
    {
        entityMethod.MakeGenericMethod(type).Invoke(modelBuilder, new object[] { });
    }
}

The code above works by identifying all non-abstract classes derived from Entity and invoking the DbModelBuilder.Entity() method for each of those classes. With this, we no longer need the DbSet property in the Context class.

Creating a repository

Without any DbSet properties on the Context, we can still access entities using the DbContext.Set method. For this tutorial, we'll do something else though: Abstract away Entity Framework by wrapping use of the Context in a repository. Using the repository will insulate the rest of your code from any changes you might want to make in you ORM implementation and will avoid tying up the rest of your project up on Entity Frameworks particulars. In a future blog post, we'll use this flexibility to demonstrate completely switching out the underlying ORM.

The repository conforms to this interface:

public interface IRepository
{
    T Get<T>(int id) where T : Entity;
    IQueryable<T> GetAll<T>() where T : Entity;
    void SaveOrUpdate<T>(T entity) where T : Entity;
    void Delete<T>(T entity) where T : Entity;
}

The full implementation is online. With that, we can simplify our HomeController to use the new repository:

public class HomeController : Controller
{
    private readonly IRepository _repository = new Repository(new Context());

    public ActionResult Index()
    {
        return View(_repository.GetAll<User>());
    }

    public ActionResult Create(User user)
    {
        _repository.SaveOrUpdate(user);

        return RedirectToAction("Index");
    }
}

And even better, we've reduced the complexity of adding new classes to the data model: You only have to add a class derived from Entity and then the reflection-based model configuration and Entity Framework migrations will automatically create the relevant schema changes. The full code sample is available online.

Yesterday, we published a blog post on how to load geographic shapefile data into SQL Server using Entity Framework. Today, we'll show how to query and display this data on a map.

Querying geographic data

Remember that the objective was to let the user click anywhere on a browser map and be shown the bondary of the Danish municipality he clicked on. The ASP.NET MVC controller action to get accomplish this looks like so:

public ActionResult Show(double latitude, double longitude)
{
    var point = DbGeography.FromText(string.Format("POINT ({0} {1})", longitude, latitude),
        4326);
    var district = _context.Features
        .Where(x => x.Geography.Intersects(point))
        .Select(x => new { Name = x.Name, Geography = SqlSpatialFunctions.Reduce(
            x.Geography, 100).AsText() })
        .SingleOrDefault();

    if (district != null)
    {
        return Json(new
        {
            name = district.Name,
            geography = district.Geography,
        }, JsonRequestBehavior.AllowGet);
    }

    Response.StatusCode = 404;
    Response.TrySkipIisCustomErrors = true;
    return Json(new { }, JsonRequestBehavior.AllowGet);
}

The beauty of using Entity Framework is that we can use this line to find features that intersect the point that the user clicked on:

.Where(x => x.Geography.Intersects(point))

Entity Framework and SQL Server have some other tricks up their sleeves. One is SqlSpatialFunctions.Reduce() which lets us reduce the number of vertices in a polygon. This turns out to be important when the underlying spatial data is very detailed but needs to betransmitted to users quickly. The polygon of a municipality, using the most accurate data from the Danish mapping authority, may take up as much as 2 MB. The Reduce function lets us trade accuracy for reduced size so that shapes can be sent to a users' browser quickly.

The last trick is the AsText() method which converts the shape to Well-known text. We can pass this to OpenLayers to draw.

Rendering geographic data

For the web frontend for our geo data, we will use OpenLayers. Openlayers is a convenient abstraction of the various web-map libraries.

Adding OpenLayers and using this HTML and CSS will get us a nice full-screen map. HTML:

<div id="map"></div>

<script src="/assets/js/openlayers/OpenLayers.js"></script>
<script src="/assets/js/jquery-1.8.1.min.js"></script>
<script type="text/javascript" src="http://maps.stamen.com/js/tile.stamen.js?v1.1.3"></script>
<script src="/assets/js/map.js"></script>

CSS:

html, body, #map {
    margin: 0;
    width: 100%;
    height: 100%;
}

The map setup in map.js is straightforward. The vector layer is where municipality shapes are drawn:

$().ready(function () {
    var map = new OpenLayers.Map({
        div: "map",
        displayProjection: "EPSG:4326",
        layers: [
            new OpenLayers.Layer.Stamen("watercolor")
        ]
    });

    var bounds = new OpenLayers.Bounds([7.81, 54.52, 12.82, 57.87]);
    map.zoomToExtent(bounds.transform(map.displayProjection, map.getProjectionObject()));

    var vector = new OpenLayers.Layer.Vector("Vector Layer", {
        projection: map.displayProjection,
        styleMap: new OpenLayers.StyleMap({'default':{
            fillColor: '#000000',
            fillOpacity: 0.5,
            label: '${name}'
        }})
    });

    map.addLayer(vector);
});

The click handler looks like this:

map.events.register("click", map, function (event) {
    var lonLat = map.getLonLatFromPixel(event.xy)
        .transform(map.getProjectionObject(), map.displayProjection);
    $.ajax({
        url: '/feature/show',
        data: {
            latitude: lonLat.lat,
            longitude: lonLat.lon
        }
    }).done(function (result) {
        var reader = new OpenLayers.Format.WKT({
            'internalProjection': map.getProjectionObject(),
            'externalProjection': map.displayProjection
        });

        var feature = reader.read(result.geography);
        feature.attributes = {
            name: result.name
        };

        vector.removeAllFeatures();
        vector.addFeatures(feature);

        map.zoomToExtent(vector.getDataExtent());
    });
});

Note that we have to make sure that coordinates passed to the server are in WGS 84 (here referred to as EPSG:4326). When we get shapes from the server, the WKT reader also transforms hte coordinates into to projection used by the map.

We use the whimsical Stamen Watercolor tile-layer for fun.

Some more setup

In addition to the setup detailed in the previous post, we'll want to add additional automation of data load and deployment environment setup. To automatically load spatial data if it's not already in the database, we'll override the Seed() method in the Entity Framework Configuration:

protected override void Seed(Context context)
{
    if (!context.Features.Any())
    {
        var assemblyDirectory =
            Path.GetDirectoryName(new Uri(Assembly.GetExecutingAssembly().CodeBase).LocalPath);
        var seedDataFilePath =
            Directory.GetFiles(assemblyDirectory, "KOMMUNE.*", SearchOption.AllDirectories).First();
        var seedDataDirectory = Path.GetDirectoryName(seedDataFilePath);

        var shapes = new DagiShapeFileReader()
            .Read(Path.Combine(seedDataDirectory, "KOMMUNE"), "KOMNAVN");

        foreach (var shape in shapes)
        {
            context.Features.Add(new Feature { Name = shape.Key, Geography = shape.Value });
        }
        context.SaveChanges();
    }

    base.Seed(context);
}

The shape files are included as content in the project that holds the entity framework setup to ensure it's available at runtime.

The last piece of housekeeping that we need to do is to make sure that the spatial assemblies (Microsoft.SqlServer.Types.dll and SqlServerSpatial110.dll) are available at runtime. The rationale behind this is descriped in this blog post, and the newest version of the dll's can be had here – you want SQLSysClrTypes.msi. In case they are not installed on the server running the app, we simply bin-deploy them using this post-build event (this assumes you're storing them in a folder called lib\Microsoft.SqlServer.Types):

copy "$(SolutionDir)lib\Microsoft.SqlServer.Types\*" "$(WebProjectOutputDir)\$(OutputPath)"

Wrap up

That's it for part 2 of this guide. You can try out the app running on AppHarbor and take a look at the full source code on GitHub. We expect to expand on the sample as we build our Entity Framework Spatial knowledge.

SQL Server has had spatial data types since 2008. Spatial data types are great if you need to manage data that involves geographic points, distances, areas and other stuff that might go on a map. With the recent release of Entity Framework 5, .NET developers now also have a convinient way to manipulate spatial data in SQL Server without writing raw SQL.

We're writing a few blog posts to show how to take advantage of this new feature and to demonstrate some of the use cases spatial support in Entity Framework 5 opens up. This is the first one, and in this post, we're covering how to load data from shape files.

Shape files are an ancient format for exchanging geographic data. It's still much used though, and if you can read Shape files, a lot geographic data is yours for the taking.

Note that Scott Hanselman already has a great blog post on how to get started with storing points with ASP.NET MVC and Entity Framework – check it out if you want a quick primer.

Initial setup

This sample is based on the Entity Framework Code First automatic migration setup that we wrote about some months ago. We'll be using Guid's for id's, which requires makes the abstract base entity class look like so:

public abstract class Entity
{
    [Key, DatabaseGenerated(DatabaseGeneratedOption.Identity)]
    public Guid Id { get; set; }
}

The concrete entity class looks like this:

public class Feature : Entity
{
    public string Name { get; set; }
    public DbGeography Geography { get; set; }
}

Entity Framework takes care of the mapping the DbGeography attribute to the correct SQL Server column type.

There's no spatial support in SQL Server Compact Edition, so you can't use the embedded approach for local development mentioned in the automatic migrations post. Instead, you'll have to connect to a local SQL Server Express instance with spatial support enabled when you're debugging.

Example Data

In this example, we'll be using data from the Danish mapping authority. They publish a great deal of data, including boundaries of the various administrative regions in Denmark. In particular, we'll load the shapes of the 98 Danish municipalities (and 1 special administrative region). This will let us figure out what municipal jurisdiction any location is under and display the boundary of that municipality on a map.

Another part of the Danish government already runs such a service (and the code is even open source), which makes this a somewhat academic, but still instructive, exercise.

Reading Shape Files

With a suitable data set and the data model in place, we can start importing data. We'll be using NetTopologySuite to parse the actual shape files. That gives us an IDataReader interface to read the shapes found in a file. In this dataset, each municipality may be represented by multiple records. In particular, a municipality made up of multiple polygons will have a record for each polygon. For that reason, we need to be able to combine multiple polygons representing one municipality into one multipolygon:

public class ShapeFileHelper
{
    public IEnumerable<KeyValuePair<string, IGeometry>> Read(string filePath,
        string nameColumnIdentifier)
    {
        return from x in ReadShapes(filePath, nameColumnIdentifier) 
               group x by x.Key into g
               select new KeyValuePair<string, IGeometry>(g.Key,
                    (IGeometry)GeometryFactory.Default.CreateMultiPolygon(g.Select(y =>
                        (IPolygon)y.Value).ToArray()));
    }

    private IEnumerable<KeyValuePair<string, IGeometry>> ReadShapes(string filePath,
        string nameColumnIdentifier)
    {
        using (var reader = new ShapefileDataReader(filePath, GeometryFactory.Default))
        {
            while (reader.Read())
            {
                yield return new KeyValuePair<string, IGeometry>(
                    (string)reader[nameColumnIdentifier], reader.Geometry);
            }
        }
    }
}

The data we're using for this sample has some other quirks that must be addressed before it can be passed to Entity Framework and SQL Server. One is the reference point ("datum") and coordinate system used to encode coordinates. A good format to store geographic data in, is WSG 84, which is what GPS devices use. Data from Danish authorities comes in a coordinate system called ETRS89 and uses the UTM 32 datum however. To transform the coordinates to WSG 84, we'll use DotSpatial which comes with NetTopologySuite. Setting up the transform looks like this:

private const string _wgs84wkt = "GEOGCS[\"WGS 84\",DATUM[\"WGS_1984\",SPHEROID[\"WGS 84\",6378137,298.257223563,AUTHORITY[\"EPSG\",\"7030\"]],AUTHORITY[\"EPSG\",\"6326\"]],PRIMEM[\"Greenwich\",0,AUTHORITY[\"EPSG\",\"8901\"]],UNIT[\"degree\",0.01745329251994328,AUTHORITY[\"EPSG\",\"9122\"]],AUTHORITY[\"EPSG\",\"4326\"]]";
private const string _etrs89utm32wkt = "PROJCS[\"ETRS89 / UTM zone 32N\",GEOGCS[\"ETRS89\",DATUM[\"D_ETRS_1989\",SPHEROID[\"GRS_1980\",6378137,298.257222101]],PRIMEM[\"Greenwich\",0],UNIT[\"Degree\",0.017453292519943295]],PROJECTION[\"Transverse_Mercator\"],PARAMETER[\"latitude_of_origin\",0],PARAMETER[\"central_meridian\",9],PARAMETER[\"scale_factor\",0.9996],PARAMETER[\"false_easting\",500000],PARAMETER[\"false_northing\",0],UNIT[\"Meter\",1]]";

private readonly DotSpatialMathTransform _transform;

public DagiShapeFileReader()
{
    var etrs89utmprojection = ProjectionInfo.FromEsriString(_etrs89utm32wkt);
    var wgs84projection = ProjectionInfo.FromEsriString(_wgs84wkt);
    _transform = new DotSpatialMathTransform(etrs89utmprojection, wgs84projection);
}

Another technicality is how SQL Server treats polygons. Imagine a polygon demarcating an area on a map. Is that supposed to mean that the area is what's inside the polygon or that it's the rest of the world except what's inside the polygon? We'll use that the heuristic that the outcome of these two interpretations with the smallest out area is likely to be correct. So for all shapes read, we invert the shape, check to see if the inverted area is smaller than the original and if so, use the inverted shape. With the transform, that comes to this:

public IEnumerable<KeyValuePair<string, DbGeography>> Read(string filePath,
    string nameColumnIdentifier)
{
    var shapes = new ShapeFileHelper().Read(filePath, nameColumnIdentifier);
    return shapes.AsParallel().Select(x =>
    {
        var transformedGeography = GeometryTransform.TransformGeometry(
            GeometryFactory.Default, x.Value, _transform);

        var sqlGeography =
            SqlGeography.STGeomFromText(new SqlChars(transformedGeography.AsText()), 4326)
            .MakeValid();

        var invertedSqlGeography = sqlGeography.ReorientObject();
        if (sqlGeography.STArea() > invertedSqlGeography.STArea())
        {
            sqlGeography = invertedSqlGeography;
        }

        var dbGeography = DbSpatialServices.Default.GeographyFromProviderValue(sqlGeography);

        return new KeyValuePair<string, DbGeography>(x.Key, dbGeography);
    });
}

Now we can finally read the shapefile shapes into SQL Server using Entity Framework (KOMNAVN is the column that holds the name of the municipality):

using (var context = new Context())
{
    foreach (var shape in new DagiShapeFileReader().Read(pathToShapeFile, "KOMNAVN"))
    {
        context.Features.Add(new Feature { Name = shape.Key, Geography = shape.Value });
    }
    context.SaveChanges();
}

The next installment will cover how to query and display the data we've read. We'll also release the full solution. In the meantime you can try out the end result here. Click anywhere on the Danish land-mass to see the boundary of the municipality you clicked on.

As of now, you can provision the StatsMix add-on for your AppHarbor applications.

StatsMix is a great way to track events and actions in your application. Even better, StatsMix is a powerful platform for visualizing data and using charts and dashboards. Metrics and dashboards can be shared via email, public urls or embedding.

To start using StatsMix on AppHarbor, simply add the add-on to your application. This will make a STATSMIX_URL configuration variable available to your application. Use this when interacting with the StatsMix REST API. John Sheehan has a good Gist demonstrating how to use StatsMix from .NET with RestSharp.

StatsMix also maintains a NuGet package and there's a .NET documentation page on their site.

A few days ago, GitHub launched a new Commit Status API. The API lets you integrate continuous integration setups to report back the status of a build to pull requests on GitHub.

We use GitHub ourselves and have been wanting something like this for a while and we immediately implemented support for commit statuses in our existing GitHub integration. We needed to add some additional interface before making the feature available, and that is why you're only getting it now.

To take advantage of commit statuses, you may have to deactivate and re-add the GitHub integration on AppHarbor. On GitHub, deactivating is done in Admin -> Service Hooks ->AppHarbor. Once that is done, you can re-add the integration from the Settings menu for your app on AppHarbor.

Over the last couple of weeks, we've been rolling out an interesting new under-the-covers feature: Web worker pre-warming.

Previously, when AppHarbor deployed a new version of your application the platform would create new workers and then immediately switch over load from old to new workers. Old workers would be kept around for a while to ensure that any in-flight requests could finish processing.

Some web applications take a while to start up before they can start processing requests, even with view pre-compilation enabled. With the old platform deployment approach, your web application would not respond to new requests until after start-up had completed on the new web workers.

The new pre-warming feature changes this: Instead of switching load immediately after deploying a new application version, AppHarbor now monitors new workers by sending requests to the site root. Only when the platform can successfully connect to all new workers is the routing layer updated to switch load. With the new pre-warming feature, updating applications running on AppHarbor is a lot smoother and we're pleased with the effects it's had on the sites we ourselves run on the platform, including appharbor.com.

You don't need to do anything to take advantage of web worker pre-warming, all deploys on AppHarbor are now done this way.

Note that when designing deployment and migrations processes for apps that run on AppHarbor, you should take into consideration that both old and new version of your code may be running on different workers while a deploy is in progress. This was also the case before (because old workers were kept around to complete outstanding requests) but the two versions will now potentially run side-by-side for a greater length of time.

Image credit: Erin Brown-John.

We're excited to announce that you can now provision the paid Professional plan of the New Relic add-on. New Relic is an advanced performance monitoring add-on that gives you detailed insights into how your application is performing on AppHarbor. In addition to the benefits of the free Standard plan, the Professional plan gets you more in-depth analysis, configurable alerts, longer data retention and many other benefits.

The plan is priced per-worker at $45 per worker per month and both web and background workers count towards the total. Remember that all charges on AppHarbor are prorated. If you only use the Professional plan for a week (or a day or an hour), you will only pay for that fraction of the total monthly plan price.

Try out the New Relic add-on on AppHarbor.

Today we're releasing version 1.1 of the AppHarbor CLI. This release brings, among other things, a new way of deploying applications to AppHarbor directly from the command line - without the use of any VCS. With the new appharbor deploy command everything in the current directory is packed and uploaded to AppHarbor. AppHarbor will then deploy the application as-is if no solution file is found, or build the application as usual if a solution file is found.

Note that you can also add the -h switch to any command to see command-specific options. When deploying applications with the CLI you can use this to exclude certain directories - for instance, it might make sense to ignore bin and obj directories to reduce the package size when uploading. This can now be accomplished like so:

appharbor deploy -e bin -e obj

When using the CLI you can link the current directory to a specific application on AppHarbor with the appharbor link command. With this release you'll also be able to use application specific commands with the -a switch to specify an application id. For example you can get the configuration variables associated with the "foo" application like so:

appharbor config -a foo

Changes with version 1.1

• Added appharbor deploy command.
• Added appharbor open which is a shortcut for opening the application on AppHarbor.
• Command specific options and the globally available -h switch for getting command help.
• Various bug fixes and optimizations

We hope you'll find good use of this new version of the AppHarbor CLI. The project is open source and available on GitHub so you're more than welcome to issue a pull request or ticket if you'd like to see another feature or improve the CLI.