Phoenix Todos - Static Assets

This post is written as a set of Literate Commits. The goal of this style is to show you how this program came together from beginning to end.

Each commit in the project is represented by a section of the article. Click each section's header to see the commit on Github, or check out the repository and follow along.

Written by Pete Corey on Aug 31, 2016.

Mix Phoenix.New

For this project we’ll be recreating Meteor’s Todos application using a React front-end and an Elixir/Phoenix powered backend. With limited experience with both React and Elixir/Phoenix, this should definitely be an interesting leaning experience.

This will be the largest literate commits project we’ve done to date, and will span several articles published over the upcoming weeks. Be sure to stay tuned for future posts!

This first commit creates a new Phoenix project called phoenix_todos.

Adding mix_test_watch

Moving forward, we’ll be writing tests for the Elixir code we create. Being the good developers that we are, we should test this code.

To make testing a more integrated process, we’ll add the mix_text_watch dependency, which lets us run the mix test.watch command to continuously watch our project for changes and re-run our test suite.

mix.exs

... {:gettext, "~> 0.9"}, - {:cowboy, "~> 1.0"}] + {:cowboy, "~> 1.0"}, + {:mix_test_watch, "~> 0.2", only: :dev}] end

mix.lock

"mime": {:hex, :mime, "1.0.1"}, + "mix_test_watch": {:hex, :mix_test_watch, "0.2.6"}, "phoenix": {:hex, :phoenix, "1.1.6"},

Hello React

Laying the groundwork for using React in a Phoenix project is fairly straight-forward.

We kick things off by installing our necessary NPM dependencies (react, react-dom, and babel-preset-react), and then updating our brunch-config.js to use the required Babel preset, and whitelisting our React NPM modules.

Once that’s finished, we can test the waters by replacing our app.html.eex layout template with a simple React attachment point:


<div id="hello-world"></div>

Finally, we can update our app.js to create and render a HelloWorld component within this new element:


class HelloWorld extends React.Component {
  render() {
    return (<h1>Hello World!</h1>)
  }
}
 
ReactDOM.render(
  <HelloWorld/>,
  document.getElementById("hello-world")
)

For a more detailed rundown of this setup process, be sure to read this fantastic article by Brandon Richey that walks you throw the process step by step.

brunch-config.js

... babel: { + presets: ["es2015", "react"], // Do not use ES6 compiler in vendor code ... npm: { - enabled: true + enabled: true, + whitelist: ["phoenix", "phoenix_html", "react", "react-dom"] }

package.json

{ - "repository": { - }, + "repository": {}, "dependencies": { "babel-brunch": "^6.0.0", + "babel-preset-react": "^6.11.1", "brunch": "^2.0.0", "javascript-brunch": ">= 1.0 < 1.8", + "react": "^15.3.1", + "react-dom": "^15.3.1", "uglify-js-brunch": ">= 1.0 < 1.8"

web/static/js/app.js

-// Brunch automatically concatenates all files in your -// watched paths. Those paths can be configured at -// config.paths.watched in "brunch-config.js". -// -// However, those files will only be executed if -// explicitly imported. The only exception are files -// in vendor, which are never wrapped in imports and -// therefore are always executed. +import React from "react" +import ReactDOM from "react-dom" -// Import dependencies -// -// If you no longer want to use a dependency, remember -// to also remove its path from "config.paths.watched". -import "deps/phoenix_html/web/static/js/phoenix_html" +class HelloWorld extends React.Component { + render() { + return (

Hello World!

) + } +} -// Import local files -// -// Local files can be imported directly using relative -// paths "./socket" or full ones "web/static/js/socket". - -// import socket from "./socket" +ReactDOM.render( + , + document.getElementById("hello-world") +)

web/templates/layout/app.html.eex

<body> - <div class="container"> - <header class="header"> - <nav role="navigation"> - <ul class="nav nav-pills pull-right"> - <li><a href="http://www.phoenixframework.org/docs">Get Started</a></li> - </ul> - </nav> - <span class="logo"></span> - </header> - - <p class="alert alert-info" role="alert"><%= get_flash(@conn, :info) %></p> - <p class="alert alert-danger" role="alert"><%= get_flash(@conn, :error) %></p> - - <main role="main"> - <%= render @view_module, @view_template, assigns %> - </main> - - </div> <!-- /container --> + <div id="hello-world"></div> <script src="<%= static_path(@conn, "/js/app.js") %>"></script>

Static Assets

Now we can start working in broad strokes. Since this project is a direct clone of the Meteor Todos application, we’re not planning on modifying the application’s stylesheets.

This means that we can wholesale copy the contents of ~/todos/.meteor/.../merged-stylesheets.css into our web/static/css/app.css file.

We can also copy all of the Meteor application’s static assets into our web/static/assets folder, and update our Phoenix Endpoint to make them accessible.

Reloading our application should show us a nice gradient background, and we shouldn’t see any Phoenix.Router.NoRouteError errors when trying to access our static assets.

lib/phoenix_todos/endpoint.ex

... at: "/", from: :phoenix_todos, gzip: false, - only: ~w(css fonts images js favicon.ico robots.txt) + only: ~w(css font js icon favicon.ico favicon.png apple-touch-icon-precomposed.png logo-todos.svg)

Layouts and Containers

Now that we have our basic React functionality set up and our static assets being served, it’s time to start migrating the React components from the Meteor Todos application into our new Phoenix application.

We’ll start this process by changing our app.html.eex file to use the expected "app" ID on its container element.


<div id="app"></div>

Next, we can update our app.js file, removing our HelloWorld component, and replacing it with the setup found in the Todos application. We need to be sure to remove the Meteor.startup callback wrapper, as we won’t be using Meteor:


import { render } from "react-dom";
import { renderRoutes } from "./routes.jsx";

render(renderRoutes(), document.getElementById("app"));

Now we port over the routes.jsx file. We’ll put this directly into our web/static/js folder, next to our app.js file.

We’ll keep things simple at first by only defining routes for the AppContainer and the NotFoundPage.


import AppContainer from './containers/AppContainer.jsx';
import NotFoundPage from './pages/NotFoundPage.jsx';

export const renderRoutes = () => (
  <Router history={browserHistory}>
    <Route path="/" component={AppContainer}>
      <Route path="*" component={NotFoundPage}/>
    </Route>
  </Router>
);

The AppContainer in the Meteor application defines a reactive container around an App component. This is very Meteor-specific, so we’ll gut this for now and replace it with a simple container that sets up our initial application state and passes it down to the App component.

Next comes the process of migrating the App component and all of its children components (UserMenu, ListList, ConnectionNotification, etc…).

This migration is fairly painless. We just need to be sure to remove references to Meteor-specific functionality. We’ll replace all of the functionality we remove in future commits.

After all of these changes, we’re greeted with a beautifully styled loading screen when we refresh our application.

package.json

"react": "^15.3.1", + "react-addons-css-transition-group": "^15.3.1", "react-dom": "^15.3.1", + "react-router": "^2.7.0", "uglify-js-brunch": ">= 1.0 < 1.8"

web/static/js/app.js

-import React from "react" -import ReactDOM from "react-dom" +import { render } from "react-dom"; +import { renderRoutes } from "./routes.jsx"; -class HelloWorld extends React.Component { - render() { - return (<h1>Hello World!</h1>) - } -} - -ReactDOM.render( - <HelloWorld/>, - document.getElementById("hello-world") -) +render(renderRoutes(), document.getElementById("app"));

web/static/js/components/ConnectionNotification.jsx

+import React from 'react'; + +const ConnectionNotification = () => ( + <div className="notifications"> + <div className="notification"> + <span className="icon-sync"></span> + <div className="meta"> + <div className="title-notification">Trying to connect</div> + <div className="description">There seems to be a connection issue</div> + </div> + </div> + </div> +); + +export default ConnectionNotification;

web/static/js/components/ListList.jsx

+import React from 'react'; +import { Link } from 'react-router'; + +export default class ListList extends React.Component { + constructor(props) { + super(props); + + this.createNewList = this.createNewList.bind(this); + } + + createNewList() { + const { router } = this.context; + // TODO: Create new list + } + + render() { + const { lists } = this.props; + return ( + <div className="list-todos"> + <a className="link-list-new" onClick={this.createNewList}> + <span className="icon-plus"></span> + New List + </a> + {lists.map(list => ( + <Link + to={`/lists/${ list._id }`} + key={list._id} + title={list.name} + className="list-todo" + activeClassName="active" + > + {list.userId + ? <span className="icon-lock"></span> + : null} + {list.incompleteCount + ? <span className="count-list">{list.incompleteCount}</span> + : null} + {list.name} + </Link> + ))} + </div> + ); + } +} + +ListList.propTypes = { + lists: React.PropTypes.array, +}; + +ListList.contextTypes = { + router: React.PropTypes.object, +};

web/static/js/components/Loading.jsx

+import React from 'react'; + +const Loading = () => ( + <img src="/logo-todos.svg" className="loading-app" /> +); + +export default Loading;

web/static/js/components/Message.jsx

+import React from 'react'; + +const Message = ({ title, subtitle }) => ( + <div className="wrapper-message"> + {title ? <div className="title-message">{title}</div> : null} + {subtitle ? <div className="subtitle-message">{subtitle}</div> : null} + </div> +); + +Message.propTypes = { + title: React.PropTypes.string, + subtitle: React.PropTypes.string, +}; + +export default Message;

web/static/js/components/MobileMenu.jsx

+import React from 'react'; + +function toggleMenu() { + // TODO: Toggle menu +} + +const MobileMenu = () => ( + <div className="nav-group"> + <a href="#" className="nav-item" onClick={toggleMenu}> + <span className="icon-list-unordered" title="Show menu"></span> + </a> + </div> +); + +export default MobileMenu;

web/static/js/components/UserMenu.jsx

+import React from 'react'; +import { Link } from 'react-router'; + +export default class UserMenu extends React.Component { + constructor(props) { + super(props); + this.state = { + open: false, + }; + this.toggle = this.toggle.bind(this); + } + + toggle(e) { + e.stopPropagation(); + this.setState({ + open: !this.state.open, + }); + } + + renderLoggedIn() { + const { open } = this.state; + const { user, logout } = this.props; + const email = user.emails[0].address; + const emailLocalPart = email.substring(0, email.indexOf('@')); + + return ( + <div className="user-menu vertical"> + <a href="#" className="btn-secondary" onClick={this.toggle}> + {open + ? <span className="icon-arrow-up"></span> + : <span className="icon-arrow-down"></span>} + {emailLocalPart} + </a> + {open + ? <a className="btn-secondary" onClick={logout}>Logout</a> + : null} + </div> + ); + } + + renderLoggedOut() { + return ( + <div className="user-menu"> + <Link to="/signin" className="btn-secondary">Sign In</Link> + <Link to="/join" className="btn-secondary">Join</Link> + </div> + ); + } + + render() { + return this.props.user + ? this.renderLoggedIn() + : this.renderLoggedOut(); + } +} + +UserMenu.propTypes = { + user: React.PropTypes.object, + logout: React.PropTypes.func, +};

web/static/js/containers/AppContainer.jsx

+import App from '../layouts/App.jsx'; +import React from 'react'; + +export default class AppContainer extends React.Component { + constructor(props) { + super(props); + this.state = { + user: undefined, + loading: true, + connected: true, + menuOpen: false, + lists: [] + }; + } + + render() { + return (<App {...this.state}/>); + } +};

web/static/js/layouts/App.jsx

+import React from 'react'; +import ReactCSSTransitionGroup from 'react-addons-css-transition-group'; +import UserMenu from '../components/UserMenu.jsx'; +import ListList from '../components/ListList.jsx'; +import ConnectionNotification from '../components/ConnectionNotification.jsx'; +import Loading from '../components/Loading.jsx'; + +const CONNECTION_ISSUE_TIMEOUT = 5000; + +export default class App extends React.Component { + constructor(props) { + super(props); + this.state = { + menuOpen: false, + showConnectionIssue: false, + }; + this.toggleMenu = this.toggleMenu.bind(this); + this.logout = this.logout.bind(this); + } + + componentDidMount() { + setTimeout(() => { + /* eslint-disable react/no-did-mount-set-state */ + this.setState({ showConnectionIssue: true }); + }, CONNECTION_ISSUE_TIMEOUT); + } + + componentWillReceiveProps({ loading, children }) { + // redirect / to a list once lists are ready + if (!loading && !children) { + const list = Lists.findOne(); + this.context.router.replace(`/lists/${ list._id }`{:.language-javascript}); + } + } + + toggleMenu(menuOpen = !Session.get('menuOpen')) { + Session.set({ menuOpen }); + } + + logout() { + Meteor.logout(); + + // if we are on a private list, we'll need to go to a public one + if (this.props.params.id) { + const list = Lists.findOne(this.props.params.id); + if (list.userId) { + const publicList = Lists.findOne({ userId: { $exists: false } }); + this.context.router.push(`/lists/${ publicList._id }`{:.language-javascript}); + } + } + } + + render() { + const { showConnectionIssue } = this.state; + const { + user, + connected, + loading, + lists, + menuOpen, + children, + location, + } = this.props; + + const closeMenu = this.toggleMenu.bind(this, false); + + // clone route components with keys so that they can + // have transitions + const clonedChildren = children && React.cloneElement(children, { + key: location.pathname, + }); + + return ( + <div id="container" className={menuOpen ? 'menu-open' : ''}> + <section id="menu"> + <UserMenu user={user} logout={this.logout}/> + <ListList lists={lists}/> + </section> + {showConnectionIssue && !connected + ? <ConnectionNotification/> + : null} + <div className="content-overlay" onClick={closeMenu}></div> + <div id="content-container"> + <ReactCSSTransitionGroup + transitionName="fade" + transitionEnterTimeout={200} + transitionLeaveTimeout={200} + > + {loading + ? <Loading key="loading"/> + : clonedChildren} + </ReactCSSTransitionGroup> + </div> + </div> + ); + } +} + +App.propTypes = { + user: React.PropTypes.object, // current meteor user + connected: React.PropTypes.bool, // server connection status + loading: React.PropTypes.bool, // subscription status + menuOpen: React.PropTypes.bool, // is side menu open? + lists: React.PropTypes.array, // all lists visible to the current user + children: React.PropTypes.element, // matched child route component + location: React.PropTypes.object, // current router location + params: React.PropTypes.object, // parameters of the current route +}; + +App.contextTypes = { + router: React.PropTypes.object, +};

web/static/js/pages/NotFoundPage.jsx

+import React from 'react'; +import MobileMenu from '../components/MobileMenu.jsx'; +import Message from '../components/Message.jsx'; + +const NotFoundPage = () => ( + <div className="page not-found"> + <nav> + <MobileMenu/> + </nav> + <div className="content-scrollable"> + <Message title="Page not found"/> + </div> + </div> +); + +export default NotFoundPage;

web/static/js/routes.jsx

+import React from 'react'; +import { Router, Route, browserHistory } from 'react-router'; + +// route components +import AppContainer from './containers/AppContainer.jsx'; +import NotFoundPage from './pages/NotFoundPage.jsx'; + +export const renderRoutes = () => ( + <Router history={browserHistory}> + <Route path="/" component={AppContainer}> + <Route path="*" component={NotFoundPage}/> + </Route> + </Router> +);

web/templates/layout/app.html.eex

<body> - <div id="hello-world"></div> + <div id="app"></div> <script src="<%= static_path(@conn, "/js/app.js") %>"></script>

Final Thoughts

This first installment of “Phoenix Todos” mostly consisted of “coding by the numbers”. Migrating over the styles, static assets, and front-end components of our Meteor application into our Phoenix application is tedious work to say the least.

Expect the excitement levels to ramp up in future posts. We’ll be implementing a Phoenix-style authentication system, replacing Meteor publications with Phoenix Channels, and re-implementing Meteor methods as REST endpoints in our Phoenix application.

This conversion process will open some very interesting avenues of comparison and exploration, which I’m eager to dive into.

Be sure to check back next week for Phoenix Todos - Part 2!

Assessing Mobile Meteor Applications

Written by Pete Corey on Aug 29, 2016.

Some Meteor applications are released solely as mobile applications. They’re intended to be experienced natively as a Cordova-powered application, rather than on the web through a browser.

From a security perspective, does this matter? Are security assessments for mobile-only applications approached differently than web-only, or web and mobile applications?

The answer to both questions is a resounding no!

Web is Always an Option

An interesting side-effect of the Meteor build process means that the “web” version of an application is always accessible, even if you intended to release it exclusively as a native mobile application.

During the mobile build process, you point your application at a hosted Meteor server. The mobile application communicates with the server, pulling down data and updated application code.


> meteor help build
...
Options:
  --server  Location where mobile builds connect to the Meteor server.
            Defaults to localhost:3000. Can include a URL scheme
            (for example, --server=https://example.com:443).

As expected, the application can also be accessed by navigating to this server URL directly with a browser.

This browser build can’t be disabled with current versions of Meteor. Trying to remove the “browser” platform results in an error:


> meteor remove-platform browser

While removing platforms:
error: browser: cannot remove platform in this version of Meteor

This means that the front-end of a Meteor application can always be seen by prying eyes.

Unzipping The Application

Let’s imagine that we’re trying to assess a Meteor mobile application called FooApp.

When we only have access to the compiled mobile application, how can we discover the Meteor server’s URL?

It turns out this is a fairly straight-forward process. We’ll dig into it for iOS applications (*.ipa archive files), but the same process applies to Android applications (*.apk archive packages).

Once we’ve downloaded FooApp through iTunes, its *.ipa file can usually be found at ~/Music/iTunes/iTunes Media/Mobile Applications/FooApp<version>.ipa.

Interestingly, iOS application archives can be unzipped using a standard archiving tool. The first step to discovering our server URL is to unzip the archive:


unzip FooApp<version>.ipa -d FooApp

We can now peruse through the contents of the FooApp bundle in the FooApp folder.

Finding the Server

Once we’ve unzipped our application, the server URL is within our reach.

To discover the server URL, open FooApp/Payload/FooApp.app/www/application/index.html. In that file, you’ll find a URL-encoded __meteor_runtime_config__ variable.

You can copy and paste that __meteor_runtime_config__ declaration into a browser console, and then print it in a more human friendly format:


__meteor_runtime_config__ = JSON.parse(decodeURIComponent("..."));
JSON.stringify(__meteor_runtime_config__, null, 2);

The result should look something like this:


{
  "meteorRelease": "METEOR@1.3.3-ddp-batching-beta.0",
  "ROOT_URL": "https://www.fooapp.com/",
  "ROOT_URL_PATH_PREFIX": "",
  "DDP_DEFAULT_CONNECTION_URL": "https://www.fooapp.com/",
  "autoupdateVersionCordova": "86e83cfe388118db86733f1333e3a2962fcad1b6",
  "appId": "ABCDEFGHIJ1234567890",
  "meteorEnv": {
    "NODE_ENV": "production"
  }
}

You’ll notice that both ROOT_URL and DDP_DEFAULT_CONNECTION_URL point to "https://www.fooapp.com/". This is the server URL that we’ve been searching for!

Navigating to the server would deliver all of the client-side code to our browser (even if it’s guarded by a Meteor.isCordova check), and give us access to call all Meteor methods and publications.

Now we can assess our mobile Meteor application just like any other Meteor application!

Meteor in Front, Phoenix in Back - Part 2

Written by Pete Corey on Aug 22, 2016.

In our last article, we transplanted the front-end of a simple Meteor example application into a Phoenix project and wired up a Blaze template to use Phoenix Channels rather than DDP.

Today we’ll be finishing our Franken-stack by replacing the hard-coded data we’re sending down to our clients with data persisted in a database. We’ll also implement the “Add Points” functionality using Channel events.

Let’s get to it!

Creating A Player Model

Before we start pulling data from a database, we need to lay some groundwork. We’ll be using Ecto to create a model of our player, and creating some seed data to initially populate our database.

In our Phoenix project directory, we’ll use mix to generate a new model for us:


mix phoenix.gen.model Player players name:string score:integer

The phoenix.gen.modal mix task will create both a PhoenixLeaderboard.Player model, and a migration file for us. The migration file will create our players table in PostgreSQL (Phoenix’s default database) when we run this command:


mix ecto.create

The out-of-the-box PhoenixLeaderboard.Player (web/models/player.ex) model is very close to what we want. It defines name as a string, score as an integer and a set of created/updated at timestamps.

The only change we need to make here is to specify how we want each Player to be serialized into JSON. We can do this by deriving the Poison.Encoder implementation:


defmodule PhoenixLeaderboard.Player do
  use PhoenixLeaderboard.Web, :model
  @derive { Poison.Encoder, only: [:id, :name, :score] }
  ...

Seeding Our Database

Now that we have a working Player model, let’s insert some seed data into our database.

By default, seeding a database in a Phoenix project is done by writing a script that manually inserts models into your repository. To insert all of our players, we could add the following to priv/repo/seeds.exs:


alias PhoenixLeaderboard.Repo
alias PhoenixLeaderboard.Player

Repo.insert! %Player{ name: "Ada Lovelace", score: 5 }
Repo.insert! %Player{ name: "Grace Hopper", score: 10 }
Repo.insert! %Player{ name: "Marie Curie", score: 15 }
Repo.insert! %Player{ name: "Carl Friedrich Gauss", score: 20 }
Repo.insert! %Player{ name: "Nikola Tesla", score: 25 }
Repo.insert! %Player{ name: "Claude Shannon", score: 30 }

We can run this seed script with the following mix task:


mix run priv/repo/seeds.exs

If all went well, all six of our players should be stored in our database!

Publishing Players

Let’s revisit the join function in our PhoenixLeaderboard.PlayersChannel (web/channels/players_channel.ex) module.

Last time, we simply returned a hard-coded list of cleaners whenever a client joined the "players" channel. Instead, let’s return all of the players stored in the database.

To shorten references, we’ll start by aliasing PhoenixLeaderboard.Repo and PhoenixLeaderboard.Player, just like we did in our seed file:


defmodule PhoenixLeaderboard.PlayersChannel do
  use Phoenix.Channel
  alias PhoenixLeaderboard.Repo
  alias PhoenixLeaderboard.Player
  ...

Now, refactoring our join function to return all players is as simple as calling Repo.all and passing in our Player model:


  def join("players", _message, socket) do
    {:ok, Repo.all(Player), socket}
  end

Looking back at our Leaderboard application, our player list should still be filled with our scientists.

Adding Points With Events

Now we get to the truly interesting part of this experiment.

In our original Meteor application, we updated each player’s score on the client and depended on DDP to propagate that change up to our server:


'click .inc': function () {
  Players.update(Session.get("selectedPlayer"), {$inc: {score: 5}});
}

Since we’re moving away from DDP, we can no longer rely on Meteor to do this for us. We’ll need to manage this update process ourselves.

Our plan for handling these updates is to push an "add_points" channel event up to the server whenever a user clicks on the .inc button:


Template.instance().channel.push("add_points", {
  id: Session.get("selectedPlayer")
});

In our PlayersChannel, we can handle any incoming "add_points" events using the handle_in function:


def handle_in("add_points", %{"id" => id}, socket) do
  player = Repo.get!(Player, id)
  Player.changeset(player, %{score: player.score + 5})
  |> Repo.update
  |> handle_player_update(socket)
end

Out logic here is fairly straightforward: get the Player with the given id, increment his score by 5, and then update the database with our changes.

The handle_player_update function handles the result of our Repo.update. If the update was successful, we’ll broadcast the "add_points" event down to all connected clients, passing the affected Player as the event’s payload:


defp handle_player_update({:ok, player}, socket) do
  broadcast! socket, "add_points", %{id: player.id, score: player.score}
  {:noreply, socket}
end

defp handle_player_update({:error, changeset}, socket) do
  {:reply, {:error, changeset}, socket}
end

The last piece of this puzzle is handling the "add_points" events we receive on the client. Every time we receive an "add_points" event from the server, we’ll want to update the provided Player in our Players Minimongo collection:


this.channel.on("add_points", (player) => {
  Players.update(player.id, {
    $set: {
      score: player.score
    }
  });
});

And that’s it!

Now if we navigate back to our Leaderboard application and start adding points to players, we’ll see their score and position change in the interface. If we connect multiple clients, we’ll see these changes in real-time as they happen.

Final Thoughts

As fun as this was, we don’t recommend you tear your Meteor application in half like we did. This was an experiment and a learning experience, not a production ready migration path.

In the future, we may investigate more reasonable and production ready migrations routes from an application built with Meteor to a Elixir/Phoenix environment. Stay tuned!

Lastly, we realized while building this Franken-stack that Meteor’s DDP and Phoenix Channels are not one-to-one replacements for each other. Try imagining how you would implement a Meteor-style pub/sub system in Channels. It’s an interesting problem, and one we’re excited to tackle in future posts.

If you want to run the Leaderboard yourself, check out the full project on GitHub. Feel free to open an issue if you have any questions, comments, or suggestions!