Meteor in Front, Phoenix in Back - Part 2

On Aug 22, 2016 by Pete Corey

In our last article, we transplanted the front-end of a simple Meteor example application into a Phoenix project and wired up a Blaze template to use Phoenix Channels rather than DDP.

Today we’ll be finishing our Franken-stack by replacing the hard-coded data we’re sending down to our clients with data persisted in a database. We’ll also implement the “Add Points” functionality using Channel events.

Let’s get to it!

Creating A Player Model

Before we start pulling data from a database, we need to lay some groundwork. We’ll be using Ecto to create a model of our player, and creating some seed data to initially populate our database.

In our Phoenix project directory, we’ll use mix to generate a new model for us:


mix phoenix.gen.model Player players name:string score:integer

The phoenix.gen.modal mix task will create both a PhoenixLeaderboard.Player model, and a migration file for us. The migration file will create our players table in PostgreSQL (Phoenix’s default database) when we run this command:


mix ecto.create

The out-of-the-box PhoenixLeaderboard.Player (web/models/player.ex) model is very close to what we want. It defines name as a string, score as an integer and a set of created/updated at timestamps.

The only change we need to make here is to specify how we want each Player to be serialized into JSON. We can do this by deriving the Poison.Encoder implementation:


defmodule PhoenixLeaderboard.Player do
  use PhoenixLeaderboard.Web, :model
  @derive { Poison.Encoder, only: [:id, :name, :score] }
  ...

Seeding Our Database

Now that we have a working Player model, let’s insert some seed data into our database.

By default, seeding a database in a Phoenix project is done by writing a script that manually inserts models into your repository. To insert all of our players, we could add the following to priv/repo/seeds.exs:


alias PhoenixLeaderboard.Repo
alias PhoenixLeaderboard.Player

Repo.insert! %Player{ name: "Ada Lovelace", score: 5 }
Repo.insert! %Player{ name: "Grace Hopper", score: 10 }
Repo.insert! %Player{ name: "Marie Curie", score: 15 }
Repo.insert! %Player{ name: "Carl Friedrich Gauss", score: 20 }
Repo.insert! %Player{ name: "Nikola Tesla", score: 25 }
Repo.insert! %Player{ name: "Claude Shannon", score: 30 }

We can run this seed script with the following mix task:


mix run priv/repo/seeds.exs

If all went well, all six of our players should be stored in our database!

Publishing Players

Let’s revisit the join function in our PhoenixLeaderboard.PlayersChannel (web/channels/players_channel.ex) module.

Last time, we simply returned a hard-coded list of cleaners whenever a client joined the "players" channel. Instead, let’s return all of the players stored in the database.

To shorten references, we’ll start by aliasing PhoenixLeaderboard.Repo and PhoenixLeaderboard.Player, just like we did in our seed file:


defmodule PhoenixLeaderboard.PlayersChannel do
  use Phoenix.Channel
  alias PhoenixLeaderboard.Repo
  alias PhoenixLeaderboard.Player
  ...

Now, refactoring our join function to return all players is as simple as calling Repo.all and passing in our Player model:


  def join("players", _message, socket) do
    {:ok, Repo.all(Player), socket}
  end

Looking back at our Leaderboard application, our player list should still be filled with our scientists.

Adding Points With Events

Now we get to the truly interesting part of this experiment.

In our original Meteor application, we updated each player’s score on the client and depended on DDP to propagate that change up to our server:


'click .inc': function () {
  Players.update(Session.get("selectedPlayer"), {$inc: {score: 5}});
}

Since we’re moving away from DDP, we can no longer rely on Meteor to do this for us. We’ll need to manage this update process ourselves.

Our plan for handling these updates is to push an "add_points" channel event up to the server whenever a user clicks on the .inc button:


Template.instance().channel.push("add_points", {
  id: Session.get("selectedPlayer")
});

In our PlayersChannel, we can handle any incoming "add_points" events using the handle_in function:


def handle_in("add_points", %{"id" => id}, socket) do
  player = Repo.get!(Player, id)
  Player.changeset(player, %{score: player.score + 5})
  |> Repo.update
  |> handle_player_update(socket)
end

Out logic here is fairly straightforward: get the Player with the given id, increment his score by 5, and then update the database with our changes.

The handle_player_update function handles the result of our Repo.update. If the update was successful, we’ll broadcast the "add_points" event down to all connected clients, passing the affected Player as the event’s payload:


defp handle_player_update({:ok, player}, socket) do
  broadcast! socket, "add_points", %{id: player.id, score: player.score}
  {:noreply, socket}
end

defp handle_player_update({:error, changeset}, socket) do
  {:reply, {:error, changeset}, socket}
end

The last piece of this puzzle is handling the "add_points" events we receive on the client. Every time we receive an "add_points" event from the server, we’ll want to update the provided Player in our Players Minimongo collection:


this.channel.on("add_points", (player) => {
  Players.update(player.id, {
    $set: {
      score: player.score
    }
  });
});

And that’s it!

Now if we navigate back to our Leaderboard application and start adding points to players, we’ll see their score and position change in the interface. If we connect multiple clients, we’ll see these changes in real-time as they happen.

Final Thoughts

As fun as this was, we don’t recommend you tear your Meteor application in half like we did. This was an experiment and a learning experience, not a production ready migration path.

In the future, we may investigate more reasonable and production ready migrations routes from an application built with Meteor to a Elixir/Phoenix environment. Stay tuned!

Lastly, we realized while building this Franken-stack that Meteor’s DDP and Phoenix Channels are not one-to-one replacements for each other. Try imagining how you would implement a Meteor-style pub/sub system in Channels. It’s an interesting problem, and one we’re excited to tackle in future posts.

If you want to run the Leaderboard yourself, check out the full project on GitHub. Feel free to open an issue if you have any questions, comments, or suggestions!

Advent of Code: Not Quite Lisp

On Aug 17, 2016 by Pete Corey

This post is written as a set of Literate Commits. The goal of this style is to show you how this program came together from beginning to end.

Each commit in the project is represented by a section of the article. Click each section's header to see the commit on Github, or check out the repository and follow along.

Christmas in August

Today’s an exciting day! We’ll be tackling an entirely new set of code katas using an entirely different language! We’ll be working on the first challenge in the Advent of Code series.

In the same vein as one of our recent posts, we’ll be using the Elixir language to solve this challenge.

This first commit is the result of mix new advent_of_code_01 and creates a base Elixir project.

config/config.exs

+use Mix.Config

lib/advent_of_code_01.ex

+defmodule AdventOfCode01 do +end

mix.exs

+defmodule AdventOfCode01.Mixfile do + use Mix.Project + + def project do + [app: :advent_of_code_01, + version: "0.1.0", + elixir: "~> 1.3", + build_embedded: Mix.env == :prod, + start_permanent: Mix.env == :prod, + deps: deps()] + end + + def application do + [applications: [:logger]] + end + + defp deps do + [] + end +end

test/advent_of_code_01_test.exs

+defmodule AdventOfCode01Test do + use ExUnit.Case + doctest AdventOfCode01 + + test "the truth" do + assert 1 + 1 == 2 + end +end

test/test_helper.exs

+ExUnit.start()

Watching Tests

Out of the box, Elixir comes with a fantastic unit testing framework. ExUnit can be run against our current project by running the mix test command. Unfortunately, this runs the test suite one time and quits. It doesn’t watch our project and rerun our suite when it detects changes.

Thankfully, the mix_test_watch dependency does exactly that. We can add it to our project, and run the mix test.watch command. Our test suite will rerun on every file change.

Now that our test suite is up and running, we’ll remove the example test Elixir provided for us.

mix.exs

... defp deps do - [] + [ + {:mix_test_watch, "~> 0.2", only: :dev} + ] end

test/advent_of_code_01_test.exs

... doctest AdventOfCode01 - - test "the truth" do - assert 1 + 1 == 2 - end end

Our First Test

To get things going, let’s write the simplest test we can think of. Amazingly, tests for Elixir functions can be written within the documentation for the function itself. For example, we can write our first test like this:


iex> AdventOfCode01.which_floor "("
1

Elixir will tease these tests out of the function docs and run them as part of our test suite.

We can make this first test pass by simply returning 1 from our new which_floor function:


def which_floor(directions) do
  1
end

And with that, our test suite flips back to green.

lib/advent_of_code_01.ex

defmodule AdventOfCode01 do + + @doc """ + Determines which floor Santa will end up on. + + ## Examples + + iex> AdventOfCode01.which_floor "(" + 1 + + """ + def which_floor(directions) do + 1 + end + end

Jumping Forward

Our next test is slightly more complicated:


iex> AdventOfCode01.which_floor "(("
2

Normally, we might take a bit more time to flesh out more naive, intemediary solutions, but this is a fairly easy problem to solve. We want to split our directions string into its component characters, map those characters into numbers (1 in this case), and then sum the result:


directions
|> String.split("", trim: true)
|> Enum.map(&handle_direction/1)
|> Enum.sum()

Our handle_direction private function expects to recieve a "(" as its input and always returns a 1:


defp handle_direction("("), do: 1

With those changes, our suite returns to green.

lib/advent_of_code_01.ex

... + iex> AdventOfCode01.which_floor "((" + 2 + """ def which_floor(directions) do - 1 + directions + |> String.split("", trim: true) + |> Enum.map(&handle_direction/1) + |> Enum.sum() end + defp handle_direction("("), do: 1 + end

Handling All Matches

Let’s add one last test that excercises the ability to process “down” directions:


iex> AdventOfCode01.which_floor("()(")
1

After adding this test, our suite fails. It complains that it can’t find a “function clause matching in AdventOfCode01.handle_direction/1”.

This is because we’re only handling “up” directions ("(") in the handle_direction function:


defp handle_direction("("), do: 1

Let’s add a new match for “down” directions:


defp handle_direction(")"), do: -1

After adding that new function definition to our module, our tests flip back to green. Victory!

lib/advent_of_code_01.ex

... + iex> AdventOfCode01.which_floor "()(" + 1 + """ ... defp handle_direction("("), do: 1 + defp handle_direction(")"), do: -1

Final Thoughts

Elixir is a beautiful language with lots of exciting features. In this example, we got a taste of transformation pipelines and how pattern matching can be used to write expressive control flow structures.

The first class treatment of documentation and testing is a welcome surprise coming from an ecosystem where testing seems to be an afterthought.

Around here we’re big fans of using small practice problems and code katas to learn new languages and paradigms. Expect to see Elixir making appearances in upcoming Literate Commit posts!

Meteor in Front, Phoenix in Back - Part 1

On Aug 15, 2016 by Pete Corey

If you follow me on Twitter, it’s probably not surprise that I’ve been interested in Elixir and the Phoenix Framework for quite a while now. Coming from a Node.js and Meteor background, the promises of out-of-the-box reliability and scalability are incredibly attractive.

To get my feet wet, I decided to use Phoenix to build out a back-end for a simple Meteor example app.

Let’s put on our mad scientist hats and build a “Franken-stack”. We’ll be tearing the front-end out of an existing Meteor application, dropping it into a new Phoenix application, rewriting a Blaze template to use Phoenix Channels instead of DDP, and then writing a simple replacement back-end in Elixir.

Let’s get started!

Building Leaderboard

The Meteor example application we’ll be using is Leaderboard. It’s a very simple application that updates a single collection over DDP. The lack of moving parts makes it an ideal candidate for this kind of experimentation.

Let’s clone Leaderboard onto our machine and run it:


git clone https://github.com/meteor/leaderboard ~/leaderboard
cd ~/leaderboard
meteor

Fantastic! Meteor has built our Leaderboard application an moved the final build bundle into ~/leaderboard/.meteor/local/build. We’ll work more with that later.

Creating Phoenix Leaderboard

Now we need to create our new Phoenix project. Unsurprisingly, we’ll be calling this new app “Phoenix Leaderboard”. Let’s use Mix to create our application:


cd ~
mix phoenix.new phoenix_leaderboard
cd ~/phoenix_leaderboard
mix ecto.create

I’ll assume that you have a very basic understanding of a Phoenix project’s structure. If you don’t, check out the official guides for a quick introduction.

Now that we have our Phoenix project, we can start up our Phoenix server:


mix phoenix.server

Navigating to http://localhost:4000 should bring you to a “Welcome to Phoenix!” page.

Front-end Transplant

Now that we’ve laid our groundwork, we can move onto the more interesting bits of this experiment. Let’s get to work transplanting our Meteor front-end into our newly created Phoenix application.

Within our ~/leaderboard/.meteor/local/build/programs folder, our Meteor application’s font-end and back-end components are separated into the web.browser and server folders respectively.

Transplanting the front-end is really just a matter of copying over all of the required files within web.browser into our Phoenix application.

From web.browser, we’ll need the merged-stylesheets.css file, the entire app folder, and the entire packages folder. Let’s copy all of these into ~/phoenix_leaderboard/priv/static:


cp merged-stylesheets.css ~/phoenix_leaderboard/priv/static/
cp -r app ~/phoenix_leaderboard/priv/static/
cp -r packages ~/phoenix_leaderboard/priv/static/

Now we need to tell our Phoenix server that these files can be served as static assets. Let’s open up our lib/phoenix_leaderboard/endpoint.ex file and add them to our Plug.Static plug:


  plug Plug.Static,
    at: "/", from: :phoenix_leaderboard, gzip: false,
    only: ~w(app packages merged-stylesheets.css css js)

The last step of this transplant is to copy over the final HTML generated by our Meteor application. Head over to view-source:http://localhost:4000/ and copy the contents of this page into web/templates/layout/app.html.eex, replacing whatever’s already there.

That’s it!

After restarting our Phoenix server and navigating to http://localhost:4000/, we should see the (playerless) Leaderboard application!

Leveraging Brunch

Now that we’ve successfully transplanted out Meteor front-end into our Phoenix application, our next step is to wire it up to our server and start passing data.

To do this, we’re going to make some minor changes to the leaderboard Blaze template.

We’re going to be writing ES6 in this project, so we’ll want this transpiled into standard ES5-style Javascript. We can use Brunch, Phoenix’s default asset pipeline, to do this for us.

To leverage Brunch, I’m going to move the contents of priv/static/app/leaderboard.js into web/static/js/app.js, overwriting the current contents of app.js. Brunch watches web/static/js/* for changes and runs them through Babel, UglifyJS, etc… before moving it to priv/static/js/app.js.

Next, I’ll remove the self-executing function wrapper around the Blaze Template code, and import Phoenix’s socket module. Our new app.js should look something like this:


import socket from "./socket";

const Players = new Mongo.Collection("players");

Template.leaderboard.helpers({
  ...

Now we’ll change our app.html.eex file to pull in /js/app.js instead of /app/template.js:


<script src='<%= static_path(@conn, "/js/app.js") %>'></script>

Now that we’ve made these changes, we should still be able to load our new Leaderboard application without any problems.

Connecting to Channels

Now that we can freely change our leaderboard template, let’s remove our dependence on DDP and fetch player data from a Phoenix Channel instead.

Our plan of attack is to subscribe to a Channel when the leaderboard template is created and upsert any published players into our client-side Players Minimongo collection. By leveraging Minimongo, we won’t have to make any changes to our existing Meteor-style template functionality.

Let’s add an onCreated handler to our leaderboard Blaze template:


Template.leaderboard.onCreated(function() {
  this.channel = socket.channel("players");
  this.channel.join()
    .receive("ok", players => {
      players.map(player => Players.upsert(player.id, player));
    })
    .receive("error", e => console.log("Unable to join", e));
});

We’re opening a connection to a "players" Channel, and on successfully joining, we’re upserting all of the players we receive from the server into our local Players collection.

To make this work, we need to add a "players" Channel on our server.


By default, Phoenix creates a socket handler for us called PhoenixLeaderboard.UserSocket (web/channels/user_socket.ex). Here, we can define our "players" channel and assign it a controller module, PhoenixLeaderboard.PlayersChannel:


channel "players", PhoenixLeaderboard.PlayersChannel

Now let’s add a simple join handler to our new PhoenixLeaderboard.PlayersChannel (web/channels/players_channel.ex):


defmodule PhoenixLeaderboard.PlayersChannel do
  use Phoenix.Channel

  def join("players", _message, socket) do
    {:ok, [
      %{ id: 1, name: "Ada Lovelace", score: 5 },
      %{ id: 2, name: "Grace Hopper", score: 10 },
      %{ id: 3, name: "Marie Curie", score: 15 },
      %{ id: 4, name: "Carl Friedrich Gauss", score: 20 },
      %{ id: 5, name: "Nikola Tesla", score: 25 },
      %{ id: 6, name: "Claude Shannon", score: 30 }
    ], socket}
  end
end

Every time a client joins the "players" channel, we’ll send them a list of players in our reply.

If we go back to our application, we’ll see that all of our players are correctly pulled from the server and rendered in order of their score!

What’s Next and Final Thoughts

In good conscience, we should reiterate that this is just an experiment. We don’t recommend tearing a Meteor application in half and dropping its front-end into another application.

That being said, the fact that this is possible is really interesting!

It’s amazing that after dropping the Meteor front-end into our Phoenix application, we can still use all of the features of Blaze templates, Minimongo, and Session variables right out of the box!

In our next post, we’ll finish up our Franken-stack by wiring our Phoenix server up to a real database and using Channel events to implement the “Add Points” functionality.

Stay tuned!