Phoenix Todos - Public and Private Lists

This post is written as a set of Literate Commits. The goal of this style is to show you how this program came together from beginning to end.

Each commit in the project is represented by a section of the article. Click each section's header to see the commit on Github, or check out the repository and follow along.

Written by Pete Corey on Nov 16, 2016.

Make Private

Now that our channel connection can be authenticated, we can gives users the ability to make their lists private.

To start, we’ll add a "make_private" channel event handler. This handler will call List.make_private and set the list’s user_id equal to the socket’s currently authenticated user:


list = get_user_id(socket)
|> List.make_private(list_id)
|> Repo.preload(:todos)

Once we’ve done that, we’ll broadcast a "update_list" event to all connected clients.

However, if a list becomes private, we’ll want to remove it from other users’ clients, instead of just showing the change. To do this, we’ll have to intercept all outbound "update_list" events:


intercept ["update_list"]

def handle_out("update_list", list, socket) do

If a user has permission to see the outgoing list, we’ll push another "update_list" event. Otherwise, we’ll push a "remove_list" event:


case List.canView?(get_user_id(socket), list) do
  true ->
    push(socket, "update_list", list)
  false ->
    push(socket, "remove_list", list)
end

After wiring up all of the necessary Redux plumbing to call our "make_private" event, the functionality it complete.

web/channels/list_channel.ex

... + intercept ["update_list"] + defp get_user_id(socket) do ... + def handle_in("make_private", %{ + "list_id" => list_id, + }, socket) do + list = get_user_id(socket) + |> List.make_private(list_id) + |> Repo.preload(:todos) + + broadcast! socket, "update_list", list + + {:noreply, socket} + end + def handle_in("delete_todo", %{ ... + def handle_out("update_list", list, socket) do + case List.canView?(get_user_id(socket), list) do + true -> + push(socket, "update_list", list) + false -> + push(socket, "remove_list", list) + end + {:noreply, socket} + end + end

web/models/list.ex

... @required_fields ~w(name incomplete_count) - @optional_fields ~w() + @optional_fields ~w(user_id) ... + def make_private(user_id, id) do + Repo.get(PhoenixTodos.List, id) + |> changeset(%{ + user_id: user_id + }) + |> Repo.update! + end + def delete_todo(todo_id) do ... + def canView?(_, %{user_id: nil}), do: true + def canView?(user_id, %{user_id: user_id}), do: true + def canView?(_, _), do: false + end

web/static/js/actions/index.js

... +export const MAKE_PRIVATE_REQUEST = "MAKE_PRIVATE_REQUEST"; +export const MAKE_PRIVATE_SUCCESS = "MAKE_PRIVATE_SUCCESS"; +export const MAKE_PRIVATE_FAILURE = "MAKE_PRIVATE_FAILURE"; + export const DELETE_TODO_REQUEST = "DELETE_TODO_REQUEST"; ... channel.on("update_list", list => { + console.log("update_list", list) dispatch(updateList(list)); ... +export function makePrivateRequest() { + return { type: MAKE_PRIVATE_REQUEST }; +} + +export function makePrivateSuccess() { + return { type: MAKE_PRIVATE_SUCCESS }; +} + +export function makePrivateFailure() { + return { type: MAKE_PRIVATE_FAILURE }; +} + +export function makePrivate(list_id) { + return (dispatch, getState) => { + const { channel } = getState(); + dispatch(makePrivateRequest()); + channel.push("make_private", { list_id }) + .receive("ok", (list) => { + dispatch(makePrivateSuccess()); + }) + .receive("error", () => dispatch(makePrivateFailure())) + .receive("timeout", () => dispatch(makePrivateFailure())); + } +} + export function deleteTodoRequest() {

web/static/js/components/ListHeader.jsx

... } else { - makePrivate.call({ listId: list._id }, alert); + this.props.makePrivate(list.id); }

web/static/js/pages/ListPage.jsx

... deleteList, + makePrivate, deleteTodo ... updateName={this.props.updateName} - deleteList={this.props.deleteList}/> + deleteList={this.props.deleteList} + makePrivate={this.props.makePrivate} + /> <div className="content-scrollable list-items"> ... }, + makePrivate: (list_id) => { + return dispatch(makePrivate(list_id)); + }, deleteTodo: (todo_id) => {

Make Public

Just as we let users make their lists private, we need to let them make their private lists public again.

We’ll do this by adding a "make_public" channel event that sets the user_id field on the specified list to nil and broadcasts an "update_list" event.


list = List.make_public(list_id)
|> Repo.preload(:todos)

broadcast! socket, "update_list", list

Unfortunately, this introduces a situation where lists are added back into the UI through a "update_list" event rather than a "add_list" event.

To handle this, we need to check if the "UPDATE_LIST" Redux reducer actually found the list it was trying to update. If it didn’t, we’ll push the list to the end of the list, adding it to the UI:


if (!found) {
  lists.push(action.list);
}

And with that, users can make their private lists public.

web/channels/list_channel.ex

... + def handle_in("make_public", %{ + "list_id" => list_id, + }, socket) do + list = List.make_public(list_id) + |> Repo.preload(:todos) + + broadcast! socket, "update_list", list + + {:noreply, socket} + end + def handle_in("delete_todo", %{

web/models/list.ex

... + def make_public(id) do + Repo.get(PhoenixTodos.List, id) + |> changeset(%{ + user_id: nil + }) + |> Repo.update! + end + def delete_todo(todo_id) do

web/static/js/actions/index.js

... +export const MAKE_PUBLIC_REQUEST = "MAKE_PUBLIC_REQUEST"; +export const MAKE_PUBLIC_SUCCESS = "MAKE_PUBLIC_SUCCESS"; +export const MAKE_PUBLIC_FAILURE = "MAKE_PUBLIC_FAILURE"; + export const DELETE_TODO_REQUEST = "DELETE_TODO_REQUEST"; ... channel.on("update_list", list => { - console.log("update_list", list) dispatch(updateList(list)); ... +export function makePublicRequest() { + return { type: MAKE_PUBLIC_REQUEST }; +} + +export function makePublicSuccess() { + return { type: MAKE_PUBLIC_SUCCESS }; +} + +export function makePublicFailure() { + return { type: MAKE_PUBLIC_FAILURE }; +} + +export function makePublic(list_id) { + return (dispatch, getState) => { + const { channel } = getState(); + dispatch(makePublicRequest()); + channel.push("make_public", { list_id }) + .receive("ok", (list) => { + dispatch(makePublicSuccess()); + }) + .receive("error", () => dispatch(makePublicFailure())) + .receive("timeout", () => dispatch(makePublicFailure())); + } +} + export function deleteTodoRequest() {

web/static/js/components/ListHeader.jsx

... if (list.user_id) { - makePublic.call({ listId: list._id }, alert); + this.props.makePublic(list.id); } else {

web/static/js/pages/ListPage.jsx

... makePrivate, + makePublic, deleteTodo ... makePrivate={this.props.makePrivate} + makePublic={this.props.makePublic} /> ... }, + makePublic: (list_id) => { + return dispatch(makePublic(list_id)); + }, deleteTodo: (todo_id) => {

web/static/js/reducers/index.js

... case UPDATE_LIST: + let found = false; let lists = state.lists.map(list => { - return list.id === action.list.id ? action.list : list; + if (list.id === action.list.id) { + found = true; + return action.list; + } + else { + return list; + } }); + if (!found) { + lists.push(action.list); + } return Object.assign({}, state, { lists });

Final Thoughts

At this point, we’ve roughly recreated all of the features of the Meteor Todos application in Phoenix and Elixir.

I’ll be the first to admit that there are many problems with the project as it currently stands. My solution to channel authentication isn’t the best, many channel events aren’t making proper authorization checks, the front-end Redux architecture is awful, etc… That being said, this was a fantastic learning experience.

Building out Meteor-esque functionality in Phoenix is definitely more work than using Meteor, but I still believe that the benefits of using an Elixir backend outweigh the drawbacks. With a little more effort, I think I’ll be able to reduce the upfront burden quite a bit through packages and libraries.

Expect many upcoming articles discussing what I’ve learned from this conversion and how to approach building Elixir and Phoenix applications from the perspective of a Meteor developer.

Basic Meteor Authentication in Phoenix

Written by Pete Corey on Nov 14, 2016.

A question that often comes up when I’m talking to Meteor developers about transitioning to Phoenix is how to handle authentication.

When transitioning, a developer with an existing application and data may want to integrate with Meteor’s existing authentication data in their Elixir/Phoenix application instead of jumping ship and switching to an entirely different authentication scheme.

Let’s dig into how Meteor’s password authentication works and how to use it within a Elixir/Phoenix application.

Setting Up Our Projects

To start, let’s assume that you have a Meteor application built with user accounts managed through the accounts-password package.

For development purposes, let’s assume that your Meteor server is running locally on port 3000, and your MongoDB database instance is running locally on port 3001.

If you want to follow along, a quick way to set this up would be to clone the example Todos application and spin it up on your machine:


git clone https://github.com/meteor/todos
cd todos
meteor

Next, register a dummy user account (e.g., "user@example.com"/"password") in your browser.


Now that Meteor has MongoDB running and populated with a Meteor-style user account, we’ll set up a new Phoenix project.

We’ll use Mix to create our application, and because we’re using MongoDB as our database, we’ll specify that we don’t want to use Ecto:


mix phoenix.new meteor_auth --no-ecto

Following the instructions in the mongodb driver package, we’ll add dependencies on the mongodb and poolboy packages, and create a MongoPool module.

Finally, we’ll add the MongoPool to our list of supervised worker processes:


children = [
  # Start the endpoint when the application starts
  supervisor(MeteorAuth.Endpoint, []),
  # Here you could define other workers and supervisors as children
  worker(MongoPool, [[database: "meteor", port: 3001]])
]

After restarting our Phoenix server, our application should be wired up and communicating with our local MongoDB database.

Anatomy of Meteor Authentication

At first glance, Meteor’s password-based authentication system can be confusing.

However, once you untangle the mess of asynchronous, highly configurable and pluggable code, you’re left with a fairly straight-forward authentication process.

Authenticating an existing user usually begins with a call to the "login" Meteor method. This method will call the login handler registered in the accounts-password package, which simply does a password check. The result of the password check is passed into the _attemptLogin function, which actually logs the user in if the password check was successful, or returns an error if the check was unsuccessful.

The results of a successful login are that the authenticated user will be associated with the current connection, and that the user’s _id, resume token, and a tokenExpires timestamp will be returned to the client.

Building an Accounts Module

To support the ability to log into a Meteor application through Elixir, we’ll build a (hugely simplified) accounts module. The module will be responsible for transforming the email and password combination passed to the server into an authenticated user session.

Let’s start by defining the module and the module’s entry points:


defmodule MeteorAuth.Accounts do

  def login(socket, %{
              "user" => %{"email" => email},
              "password" => password
            }) when is_binary(email) and is_binary(password) do
    socket
    |> attempt_login(%{query: %{"emails.0.address": email}}, password)
  end

end

The login function in our MeteorAuth.Accounts module will take in a Phoenix channel socket and a map that holds the user’s provided email address and password.

Notice that we're asserting that both email and password should be "binary" types? This helps prevent NoSQL injection vulnerabilities.

The login function calls attempt_login, which grabs the user from MongoDB based on the constructed query (get_user_from_query), checks the user’s password (valid_credentials?), and finally attempt to log the user in (log_in_user):


defp attempt_login(socket, %{query: query}, password) do
  user = get_user_from_query(query)
  valid? = valid_credentials?(user, password)
  log_in_user(valid?, socket, user)
end

To fetch the user document from MongoDB, we’re running a find query against the "users" collection, transforming the resulting database cursor into a list, and then returning the first element from that list:


defp get_user_from_query(query) do
  MongoPool
  |> Mongo.find("users", query)
  |> Enum.to_list
  |> List.first
end

To check the user’s password, we transform the user-provided password string into a format that Meteor’s accounts package expects, and then we use the Comeonin package to securely compare the hashed version of the password string with the hashed password saved in the user’s document:


defp valid_credentials?(%{"services" => %{"password" => %{"bcrypt" => bcrypt}}},
                        password) do
  password
  |> get_password_string
  |> Comeonin.Bcrypt.checkpw(bcrypt)
end

Notice how we’re using pattern matching to destructure a complex user document and grab only the fields we care about. Isn't Elixir awesome?

Before Bcrypt hashing a password string, Meteor expects it to be SHA256 hashed and converted into a lowercased base16 (hexadecimal) string. This is fairly painless thanks to Erlang’s :crypto library:


defp get_password_string(password) do
  :crypto.hash(:sha256, password)
  |> Base.encode16
  |> String.downcase
end

Our valid_credentials? function will return either a true or a false if the user-provided credentials are correct or incorrect.

We can pattern match our log_in_user function to do different things for valid and invalid credentials. If a user has provided a valid email address and password, we’ll log them in by assigning their user document to the current socket:


defp log_in_user(true, socket, user) do
  auth_socket = Phoenix.Socket.assign(socket, :user, user)
  {:ok, %{"id" => user["_id"]}, auth_socket}
end

For invalid credentials, we’ll simply return an error:


defp log_in_user(false, _socket, _user) do
  {:error}
end

Logging in Through Channels

Now that our MeteorAuth.Accounts module is finished up, we can wire it up to a Phoenix channel to test the end-to-end functionality.

We’ll start by creating a "ddp" channel in our default UserSocket module:


channel "ddp", MeteorAuth.DDPChannel

In our MeteorAuth.DDPChannel module, we’ll create a "login" event handler that calls our MeteorAuth.Accounts.login function:


def handle_in("login", params, socket) do
  case MeteorAuth.Accounts.login(socket, params) do
    {:ok, res, auth_socket} ->
      {:reply, {:ok, res}, auth_socket}
    {:error} ->
      {:reply, {:error}, socket}
  end
end

If login returns an :ok atom, we’ll reply back with an :ok status and the results of the login process (the user’s _id).

If login returns an :error, we’ll reply back to the client with an error.

To make sure that everything’s working correctly, we can make another event handler for a "foo" event. This event handler will simply inspect and return the currently assigned :user on the socket:


def handle_in("foo", _, socket) do
  user = socket.assigns[:user] |> IO.inspect
  case user do
    nil ->
      {:reply, :ok, socket}
    %{"_id" => id} ->
      {:reply, {:ok, %{"id" => id}}, socket}
  end
end

On the client, we can test to make sure that everything’s working as expected by running through a few different combinations of "foo" and "login" events:


let channel = socket.channel("ddp", {})
channel.join()

channel.push("foo")
    .receive("ok", resp => { console.log("foo ok", resp) })
    .receive("error", resp => { console.log("foo error", resp) })

...

channel.push("login", {user: {email: "user@example.com"}, password: "password"})
    .receive("ok", resp => { console.log("login ok", resp) })
    .receive("error", resp => { console.log("login error", resp) })

channel.push("foo")
    .receive("ok", resp => { console.log("foo ok", resp) })
    .receive("error", resp => { console.log("foo error", resp) })

And as expected, everything works!

We can now check if a user is currently authenticated on a socket by looking for the assigned :user. If none exists, the current user is unauthenticated. If :user exists, we know that the current user has been authenticated and is who they say they are.

Future Work

So far, we’ve only been able to log in with credentials set up through a Meteor application. We’re not creating or accepting resume tokens, and we’re missing lots of functionality related to signing up, logging out, resetting passwords, etc…

If your goal is to recreate the entirety of Meteor’s accounts package in Elixir/Phoenix, you have a long march ahead of you. The purpose of this article is to simply show that it’s possible and fairly painlessly to integrate these two stacks together.

It’s important to know that for green-field projects, or projects seriously planning on doing a full Elixir/Phoenix transition, there are better, more Phoenix-centric ways of approaching and handling user authentication and authorization.

That being said, if there’s any interest, I may do some future work related to resume tokens, signing up and out, and potentially turning this code into a more full-fledged Elixir package.

For now, feel free to check out the entire project on GitHub to get the full source. Let me know if there’s anything in particular you’d like to see come out of this!

Phoenix Todos - Authorized Sockets

This post is written as a set of Literate Commits. The goal of this style is to show you how this program came together from beginning to end.

Each commit in the project is represented by a section of the article. Click each section's header to see the commit on Github, or check out the repository and follow along.

Written by Pete Corey on Nov 9, 2016.

Authenticated Sockets

Now that we’ve implemented the bulk of the unauthenticated functionality in our application, we need to turn our attention to authenticated functionality.

To do that, we’ll need to authenticate the channel we’re using to communicate with the client. We can do this with a custom connect function that verified the user’s provided guardian_token:


def connect(%{"guardian_token" => jwt}, socket) do
  case sign_in(socket, jwt) do
    {:ok, authed_socket, guardian_params} ->
      {:ok, authed_socket}
    _ ->
      {:ok, socket}
  end
end

If the user is authenticated correctly, we’ll swap their socket out for an auth_socket, which can be used to access the current user’s object and claims.

If the user doesn’t provide a guardian_token, we’ll fall back to our old connect function:


  def connect(_params, socket) do
    {:ok, socket}
  end

All of the old functionality will continue to work as expected.

web/channels/user_socket.ex

... use Phoenix.Socket + import Guardian.Phoenix.Socket ... # performing token verification on connect. + def connect(%{"guardian_token" => jwt}, socket) do + case sign_in(socket, jwt) do + {:ok, authed_socket, _guardian_params} -> + {:ok, authed_socket} + _ -> + {:ok, socket} + end + end + def connect(_params, socket) do

Guardian Token

Now that our socket connection is expecting a guardian_token parameter, we need to supply it in our connectSocket action.

web/static/js/actions/index.js

... params: { - token: jwt + guardian_token: jwt }

Connect Socket Thunk

Because our entire socket connection will be authenticated or unauthenticated, we need to prepare ourselves to re-establish the connection every time we log in/out.

To start, we’ll need to clear out our local set of lists every time we connect to the socket.

We also need to trigger a call to our joinListsChannel thunk every time we connect.

web/static/js/actions/index.js

... export function connectSocket(jwt) { - let socket = new Socket("/socket", { - params: { - guardian_token: jwt - } - }); - socket.connect(); - return { type: CONNECT_SOCKET, socket }; + return (dispatch, getState) => { + let socket = new Socket("/socket", { + params: { + guardian_token: jwt + } + }); + socket.connect(); + dispatch({ type: CONNECT_SOCKET, socket }); + dispatch(joinListsChannel("lists.public")); + }; }

web/static/js/app.js

... store.dispatch(connectSocket(store.getState().jwt)); -store.dispatch(joinListsChannel("lists.public"));

web/static/js/reducers/index.js

... case CONNECT_SOCKET: - return Object.assign({}, state, { socket: action.socket }); + return Object.assign({}, state, { socket: action.socket, lists: [] }); case JOIN_LISTS_CHANNEL_SUCCESS:

Reconnect

Now we’ll reconnect to our socket every time a user signs in, signs up, or signs out. This ensures that the socket connection is always properly authenticated.

These changes also introduced a few small bugs which we quickly fixed.

web/static/js/actions/index.js

... dispatch(signUpSuccess(res.user, res.jwt)); + dispatch(connectSocket(res.jwt)); return true; ... dispatch(signOutSuccess()); + dispatch(connectSocket(res.jwt)); return true; ... dispatch(signInSuccess(res.user, res.jwt)); + dispatch(connectSocket(res.jwt)); return true;

web/static/js/layouts/App.jsx

... import { signOut, createList } from "../actions"; +import _ from "lodash"; ... const list = _.find(this.props.lists, list => list.id == this.props.params.id); - if (list.user_id) { + if (list && list.user_id) { const publicList = _.find(this.props.list, list => !list.user_id);

Fetching All Lists

To make things simpler, and to show off a feature of Phoenix, I’ve decided to merge the "lists.public" and "lists.private" publications into a single channel: "lists".

This channel will return all lists accessible by the current user, based on their authenticated socket.

We replaced the List.public function with List.all, which takes in a user_id. When user_id is nil, we return all public lists, as before. However, when user_id isn’t nil, we return all lists owned by that user (^user_id == list.user_id), and all public lists.

test/models/list_test.exs

... - test "public" do + test "all" do user = User.changeset(%User{}, %{ ... }) - Repo.insert!(%List{ + |> Repo.preload(:todos) + private = Repo.insert!(%List{ name: "private", ... }) + |> Repo.preload(:todos) - lists = List |> List.public |> Repo.all + public_lists = List |> List.all(nil) |> Repo.all |> Repo.preload(:todos) + all_lists = List |> List.all(user.id) |> Repo.all |> Repo.preload(:todos) - assert lists == [public] + assert public_lists == [public] + assert all_lists == [public, private] end

web/channels/list_channel.ex

... - def join("lists.public", _message, socket) do - lists = List |> List.public |> Repo.all + defp get_user_id(socket) do + case Guardian.Phoenix.Socket.current_resource(socket) do + user -> + user.id + _ -> + nil + end + end + + def join("lists", _message, socket) do + lists = List |> List.all(get_user_id(socket)) |> Repo.all {:ok, lists, socket}

web/channels/user_socket.ex

... # channel "rooms:*", PhoenixTodos.RoomChannel - channel "lists.public", PhoenixTodos.ListChannel + channel "lists", PhoenixTodos.ListChannel

web/models/list.ex

... - def public(query) do + def all(query, nil) do from list in query, ... + def all(query, user_id) do + from list in query, + where: ^user_id == list.user_id or is_nil(list.user_id), + order_by: list.inserted_at, + preload: [:todos] + end + def findByName(query, name) do

web/static/js/actions/index.js

... dispatch({ type: CONNECT_SOCKET, socket }); - dispatch(joinListsChannel("lists.public")); + dispatch(joinListsChannel("lists")); };

Final Thoughts

In hashing out this authorization scheme, I’ve realized there are lots of problems with this approach. Splitting communication across both WebSockets and REST endpoints creates lots of confusion around a user’s authorization state.

In hindsight, it would have been better to do everything over WebSockets and forget the REST user and sessions endpoints altogether. I’ll be sure to write up my thoughts around the problems with this kind of authorization and how to to it better in the future.

Next week, we should be able to finish up all authenticated functionality and finish up the Meteor to Phoeix migration project!