Automate AppCache offline support in your Webpack build

webpack-logo

Why would you use AppCache ? An API that is messy, not as advanced as service-workers and moreover, which is being removed from the Web Standards ?…

With the Progressive Web Apps, we hear a lot about service-workers. They are very powerfull for a lot of things (including offline support). Though, they’re not supported on IE nor Safari … 🙁

So, until the rest of the browser vendors catch up, if you want to provide some offline experience to all your users, you’ll have to use AppCache which is still widely supported.

AppCache in a few words

  • You have to provide a manifest.appcache file, served with the content type text/cache-manifest
  • This file will consist of three different parts:
    • CACHE: files that will be explicitly cached after they’re downloaded for the first time (this is the default section)
    • NETWORK: white-listed resources that require a connection to the server
    • FALLBACK: fallback pages the browser should use if a resource is inaccessible
  • You will reference this manifest.appcache as an attribute on the html tag of the page that will use it
  • If the manifest.appcache file is updated, the browser will download the resources listed in this manifest (if not, or offline, it will use the cached resources)

More infos on MDN

AppCache in a SPA

Note: Skip this part if you don’t bother about providing different index.html file whether your users are online or offline.

If you’re developing a SPA, and you want to provide a different index.html entry point whether you are online or offline, you’ll have to use a little trick. You won’t reference your manifest.appcache directly in your index.html but in an other html file that you’ll include in an iframe to your index.html.

That way, the index.html file won’t be cached by default (as the master entry) and you’ll be able to define a fallback in the manifest.appcache

index.html

...
<iframe src="./iframe-inject-appcache-manifest.html" style="display: none"></iframe>
...

iframe-inject-appcache-manifest.html

<html manifest="manifest.appcache"></html>

dummy manifest.appcache file

CACHE MANIFEST
# v1 (some version id)

assets/foo.png
assets/bundle.js
assets/style.css
# more assets ...

NETWORK:
*

# that way, you'll be able to force the fallback of index.html
# to an other file when you're offline
FALLBACK:
. offline.html

Automate AppCache

If your app relies on a large code base, with a build step, you will need to automate this task. You’ll also have to ensure that AppCache doesn’t mess with your development workflow (meaning disabling it when developing).

I will describe the steps I took to automate AppCache support on topheman/rxjs-experiments, a little project using RxJS (no other frameworks involved). The workflow of this project is based on a seed I made and open-sourced: topheman/webpack-babel-starter.

Checkout the App

Step 1 – Define if we should “activate” AppCache

When you’re running webpack in dev-server mode, that means you’re developing, so you want your sources to be kept up to date (you don’t want AppCache to cache them).

webpack.config.js

const MODE_DEV_SERVER = process.argv[1].indexOf('webpack-dev-server') > -1 ? true : false;
// ...
const APPCACHE = process.env.APPCACHE ? JSON.parse(process.env.APPCACHE) : !MODE_DEV_SERVER;// if false, nothing will be cached by AppCache

Continue reading

D3 React Components with Victory – Reusability / Composability

react-logo-1000-transparentlogo_d3-svg

On the topheman/d3-react-experiments project, at the end of last month, I released a few examples of React components exposing charts based either on vanilla d3 code or react-faux-dom.

In this new release, I took a full JSX approach, using some components from the Victory library. I’ve already used some of them in the past (simple pie charts / histograms), but this time, I made a more advanced chart:

  • dual Y axis (two Y axis with independent scale) / multi lines
  • user interractions:
    • mouse:
      • hover legends or lines will highlight relevent lines and axis
      • hover the chart will show a tooltip with some contextual infos
    • touch: toggle touch legends or lines will highlight relevent lines and axis
  • fully responsive
  • data comes from the npm registry API:
    • a simple configuration object will setup all the charts

Test the Demo!

Declarative

When you come back to some d3 code after a while, it can be hard to dive back in (even if it’s yours). This is where the declarative approach of React/JSX makes sense. It is easier to read and understand what’s happening, since you only have to deal with components that you feed with some props.

The code might be easier to reason about, but on the other hand, you’ll have to change your vanilla d3 approach to a component based one (and some times directly deal with svg via JSX).

Reusability / Composability

When I started doing datavisualisation in React, I felt like each chart component library I tested was too weak / not composable enough – that all you could do eventually was a single pie chart, bar chart or line chart. I almost started my own library but didn’t go through when I realised that all I would have done was making exactly the same components as libraries like Victory …

In fact, you can compose those components with each others to make more advanced charts. That way, you benefit from already available reusable components. You can even break those advanced components in multiple reusable ones on your end.

Limits

In that case, there is a limit to reusability / composability. Like any framework which gives you high level abstractions, there will come a point when your needs will become very specific and won’t fit in. If you want to make some hard core visualisations like some of which you can see on bl.ocks.org, you’ll be better coding them in vanilla d3 encapsulated in a React component.

Conclusion

  • You shouldn’t need to re-invent the wheel. React datavisualisation libraries like Victory provide components like VictoryAxis, VictoryBar, VictoryLine … They will cover your basic use cases, out of the box.
  • Those components can be composed as HOC to make advanced charts. We need people to create examples.
  • Those libraries may not be suited to make complex charts. In that case, some good ole vanilla d3, embeded in a React component will be a better solution.

Tophe

Test the Demo!

Resources:

Plain d3 code and React working together

react-logo-1000-transparentlogo_d3-svg

A few months ago, I released the v1 of topheman/d3-react-experiments. At that time, my goal was to experiment some of the existing d3 libraries that you could directly use as React components. I managed to setup a few examples with Victory and d3act.

For the v2, I took the approach of a pure d3 user that may or may not know React but would want to keep full control over the creation of his chart, using the d3 API – mixed with React lifecycle hooks.

Test the Demo!

D3 / StaticMultiLineChart

That way, I created the StaticMultiLineChart component which embeds plain d3 code (I nearly made a full copy/paste of this bl.ocks.org example) and it works just out of the box with React, without needing to know much about React lifecycle hooks.

As you can see, we only need a componentWillUpdate() lifecycle hook, to cleanup the svg node, so that drawLineChart() will work properly on each update (since DOMNodes are not reused and it appends some on each update, we would end up with multiple charts).

This is a very naive approach, but it shows how, with little knowing about React lifcecyle hooks and JSX, you can still do d3 … Transitions may not be possible to implement that way (see TransitionMultiLineChart).

View source code on Github

import React from 'react';

import ColorHash from 'color-hash';

import { scaleLinear } from 'd3-scale';
import { line } from 'd3-shape';
import { select } from 'd3-selection';
import { axisBottom, axisLeft } from 'd3-axis';

const colorHash = new ColorHash();

export default class StaticMultiLineChart extends React.Component {

  static propTypes = {
    margin: React.PropTypes.object,
    width: React.PropTypes.number,
    height: React.PropTypes.number,
    data: React.PropTypes.object.isRequired,
    minX: React.PropTypes.number,
    maxX: React.PropTypes.number,
    minY: React.PropTypes.number,
    maxY: React.PropTypes.number
  }

  static defaultProps = {
    margin: {
      top: 20,
      right: 20,
      bottom: 30,
      left: 50
    },
    width: 700,
    height: 400
  }

  constructor() {
    super();
  }

  /**
   * This example is a reuse of some plain code from an example on https://bl.ocks.org/d3noob/4db972df5d7efc7d611255d1cc6f3c4f
   * Since the render method contains .append() invocations, I remove any child of the root node at each render
   *
   * See the other examples for smarter approaches
   */
  componentWillUpdate() {
    // each update, flush the nodes of the chart - this isn't the best way - see the other example for better practice
    while (this.rootNode.firstChild) {
      this.rootNode.removeChild(this.rootNode.firstChild);
    }
  }

  drawLineChart() {
    // ... the d3 code manipulating the DOM Node this.rootNode goes here
  }

  render() {
    // only start drawing (accessing the DOM) after the first render, once we get hold on the ref of the node
    if (this.rootNode) {
      this.drawLineChart();
    }
    else {
      // setTimeout necessary for the very first draw, to ensure drawing using a DOMNode and prevent the following error:
      // "Uncaught TypeError: Cannot read property 'ownerDocument' of null"
      setTimeout(() => this.drawLineChart(), 0);
    }

    return (
       this.rootNode = node}>
    );
  }

}

D3 / TransitionMultiLineChart

To apply d3 transitions, you won’t be able to flush all the DOMNodes on each updates, you’ll need to keep a reference to each of them inside the component to let d3 mutate them (this isn’t specific to React).

We will use componentDidMount() hook to call our init() method which will create the nodes for the axis and line groups and store them on the component instance. Thanks to componentDidMount() behavior, it will only be called once and ensure we have access to the DOM created by React.

All our d3 code updating the DOM is in update() – split in updateSize() and updateData(). This update() method is called in componentDidUpdate(), which fires when props or state have changed (such as data, width, height …).

Since componentDidUpdate() is not called at first render, we call some setState() in componentDidMount() to ensure that our chart updates after the very first render.

To avoid some unnecessary DOMNodes attributes changing, we use componentWillReceiveProps() hook to check whether if it’s worth it to call updateSize() inside update() on the next tick by tagging this.shouldUpdateSize.

In fact, the difference between this code and plain d3 is only a little management of the lifecycle hooks of React to ensure you have access to you DOMNode or know when your data changed …

View source code on Github

import React from 'react';

import ColorHash from 'color-hash';

import { scaleLinear } from 'd3-scale';
import { line } from 'd3-shape';
import { select } from 'd3-selection';
import { axisLeft, axisBottom } from 'd3-axis';
import 'd3-transition';

const colorHash = new ColorHash();

export default class TransitionMultiLineChart extends React.Component {

  static propTypes = {
    margin: React.PropTypes.object,
    width: React.PropTypes.number,
    height: React.PropTypes.number,
    data: React.PropTypes.object.isRequired,
    minX: React.PropTypes.number,
    maxX: React.PropTypes.number,
    minY: React.PropTypes.number,
    maxY: React.PropTypes.number
  }

  static defaultProps = {
    margin: {
      top: 20,
      right: 20,
      bottom: 30,
      left: 50
    },
    width: 700,
    height: 400
  }

  constructor() {
    super();
    this.shouldUpdateSize = true;
    // minimal state to manage React lifecycle
    this.state = {
      initialized: false
    };
  }

  /**
   * From React doc https://facebook.github.io/react/docs/component-specs.html#mounting-componentdidmount :
   *
   * "Invoked once, only on the client (not on the server), immediately after the initial rendering occurs.
   * At this point in the lifecycle, you can access any refs to your children (e.g., to access the underlying DOM representation).
   * The componentDidMount() method of child components is invoked before that of parent components."
   *
   * this.init is called here because:
   * - we need the ref to the svg node
   * - it won't we called again
   */
  componentDidMount() {
    console.log('componentDidMount');
    this.init();
    // the code bellow is to trigger componentDidUpdate (which is not called at first render)
    setTimeout(() => {
      this.setState({
        initialized: true
      });
    });
  }

  /**
   * From React doc : https://facebook.github.io/react/docs/component-specs.html#updating-componentwillreceiveprops
   *
   * "Invoked when a component is receiving new props. This method is not called for the initial render.
   * Use this as an opportunity to react to a prop transition before render() is called
   * by updating the state using this.setState().
   * The old props can be accessed via this.props. Calling this.setState() within this function will not trigger an additional render."
   *
   * I use this hook to check whether or not this.updateSize should be called on the next update
   * Doing the same thing about this.updateData would involve deep checking the whole data passed.
   */
  componentWillReceiveProps({ margin, width, height, minX, maxX, maxY }) {
    console.log('componentWillReceiveProps');
    if (margin !== this.props.margin || width !== this.props.width || height !== this.props.height ||
      minX !== this.props.minX || maxX !== this.props.maxX || maxY !== this.props.maxY) {
      console.log('change size');
      this.shouldUpdateSize = true;
    }
  }

  /**
   * From React doc https://facebook.github.io/react/docs/component-specs.html#updating-componentdidupdate :
   *
   * "Invoked immediately after the component's updates are flushed to the DOM.
   * This method is not called for the initial render.
   * Use this as an opportunity to operate on the DOM when the component has been updated."
   *
   * this.update is called here because:
   * - it's not called for initial render - componentDidMount ensures to have our svg element init
   * - it's called after each update of the component - we get the new props
   */
  componentDidUpdate() {
    console.log('componentDidUpdate');
    this.update();
  }

  extractSize() {
    const { margin, width: widthIncludingMargins, height: heightIncludingMargins } = this.props;
    const width = widthIncludingMargins - margin.left - margin.right;
    const height = heightIncludingMargins - margin.top - margin.bottom;
    return {
      width,
      height,
      margin
    };
  }

  /**
   * Create svg nodes in order to reuse them
   */
  init() {
    console.log('init');
    this.lineGroup = this.rootNode.append('g');
    this.axisLeftGroup = this.lineGroup.append('g');
    this.axisBottomGroup = this.lineGroup.append('g');
  }

  updateSize() {
    console.log('updateSize');
    const { width, height, margin } = this.extractSize();
    const { minX, maxX, maxY } = this.props;

    // resize/re-align root nodes
    this.rootNode
      .attr('width', width + margin.left + margin.right)
      .attr('height', height + margin.top + margin.bottom);
    this.lineGroup
      .attr('transform',
        'translate(' + margin.left + ',' + margin.top + ')');

    // set domain for axis
    const xScale = scaleLinear().range([0, width]);
    const yScale = scaleLinear().range([height, 0]);

    // Scale the range of the data
    xScale.domain([minX, maxX]);
    yScale.domain([0, maxY]);

    // Update the X Axis
    this.axisBottomGroup.transition()
      .attr('transform', 'translate(0,' + height + ')')
      .call(axisBottom(xScale).ticks(width > 500 ? Math.floor(width / 80) : 4)); // prevent from having too much ticks on small screens

    // Update the Y Axis
    this.axisLeftGroup.transition()
      .call(axisLeft(yScale));

    // this.line is not called directy since it's used as a callback and is re-assigned. It is wrapped inside this.lineReference
    this.line = line() // .interpolate("monotone")
      .x(d => xScale(d.x))
      .y(d => yScale(d.y));
  }

  updateData() {
    console.log('updateData');
    const { data } = this.props;

    const drawLine = this.line;

    // prepare data to [ [{x, y, color}, {x, y, color}], [{x, y, color}, {x, y, color}] ... ]
    const processedData = [];
    Object.keys(data).forEach(countryName => {
      processedData.push(data[countryName].map((infos) => ({ color: colorHash.hex(countryName), ...infos})));
    });

    // generate line paths
    const lines = this.lineGroup.selectAll('.line').data(processedData);

    // [Update] transition from previous paths to new paths
    this.lineGroup.selectAll('.line')
      .transition()
      .style('stroke', d => d[0] ? d[0].color : null)
      .attr('d', drawLine);

    // [Enter] any new data
    lines.enter()
      .append('path')
      .attr('class', 'line')
      .style('stroke-width', '2px')
      .style('fill', 'none')
      .style('stroke', d => d[0] ? d[0].color : null)
      .attr('d', drawLine);

    // [Exit]
    lines.exit()
      .remove();
  }

  update() {
    console.log('update');
    // only call this.updateSize() if some props involving size have changed (check is done on componentWillReceiveProps)
    if (this.shouldUpdateSize === true) {
      this.updateSize();
      this.shouldUpdateSize = false;
    }
    this.updateData();
  }

  render() {
    console.log('render');
    return (
       this.rootNode = select(node)}>
    );
  }

}

Conclusion

I just wanted to expose how I managed to make d3 work with React and how, even if you’re a pure d3 user, you should still be able to take advantage of the whole possibilities of d3.

Though, with its component approach and its JSX declaration syntax, React has a great potential for reusable chart components (that’s where it becomes very interesting).

The next step on topheman/d3-react-experiments will be to try to make reusable charts based on JSX (leave the computation part to d3 and make the rendering in React, without letting d3 mutate the DOM).

This may seem like rebuilding the wheel, there are already libraries that do that kind of thing – I’ve even tested some of them on that project – but I can’t seem to find any clear winner …

Are you making data visualisations on React ? What libraries are you using ? Do you use home-made components ?…

Tophe

ES6+ code coverage with Babel plugin

babel-logo

Running unit tests against an ES6+ source code base has now become an almost trivial task, thanks to Babel and all the ecosystem around it. There are a lot of good resources explaining how to do that, with different tools and frameworks.

On the other hand, code coverage on this kind of tests is a rather less covered subject …

If you apply regular code coverage solutions to that case, you end up with reports based on the transpiled code, not your original source code in ES6+, which isn’t really relevant …

Here comes isparta

This problem was solved by isparta, a code coverage tool for ES6+, using Babel, which provides code coverage reports using istanbul (which is also a code coverage tool … 😉 more infos here).

Using isparta, you can generate code coverage from unit tests on ES6+ source code base, directly against your original source code (not the transpiled one). This works great, combined with tools like karma-coverage, you can output coverage reports under any format (html, lcov … that can be processed by various tools/services like Jenkins or coveralls.io …).

isparta not maintained anymore

No Maintenance Intended

About two weeks ago, @duglasduteil added the “No Maintenance Intended” badge to his module. This should remind us that behind every open source project there are developers maintaining them (most of the time on there free time).

So, now may be the time to find a replacement for isparta … I was looking for an alternative for a few days when I eventually ended up on twitter with @kentcdodds who mentionned dtinth/babel-plugin-__coverage__.

ES6+ code coverage using a Babel plugin

Since Babel v6, you can make your very own plugins that can do much more than simply transpile ES6+ to ES5.

Babel is a generic multi-purpose compiler for JavaScript. More than that it is a collection of modules that can be used for many different forms of static analysis.

Babel Plugin Handbook

That’s what @dtinth has done with babel-plugin-__coverage__. He made a babel plugin that instrument your code, injecting metadata that will be processed by istambul (which is used under the hood by karma-coverage and other coverage report tools).

The great thing is that, since you go through Babel anyway to transpile your source code, the only thing you have to do to get those infos to feed tools like karma-coverage (or any istambul-based coverage reporter) is to activate this plugin in your .babelrc file …

Using a babel plugin for coverage is a no-brainer.

@kentcdodds

You won’t have to add complex configuration or tools anymore. @dtinth‘s plugin is 300 LOC, it’s his first babel plugin that he made over two nights. As far as I have tested it, it works very well.

This is clear that using a babel plugin for that kind of purpose is the right way to proceed. Since it’s becoming easier to setup, we might see more projects using ES6+ including code coverage reports in a near futur.

You can see an example of setup on my project topheman/react-es6-redux.

Checkout it out

RxJS – first steps

rxjs-logo

I’ve been hearing about RxJS for a little more than 2 years in meetups and conferences, but never took the time to test it in a project (lots of other things came before on my list 😉 ).

At the end of last month, I decided to try it after watching the CycleJS Courses by André Staltz. Even if you’re more into React/Redux (like I am), I recommend it.

Before starting, I created a boilerplate for Webpack / Babel projects, gathering configurations, development & build workflow, good practices I used to setup each time I was creating one of my projects using Webpack. That way, everything would be in one place (and could be shared).

One thing RxJS is very good at is managing events (and multiple events) that have intermediate states (such as drag’n drop). So you will find the following features on topheman/rxjs-experiments (at least for the moment):

I could have coded those features in VanillaJS, but when you’ll take a look at the code, you’ll see how much simplier it is in RxJS.

Test the Demo!

I’m still a beginner at reactive programming. I can understand how it simplifies async event management. The number of operators is overwhelming though … It can be hard to find the right ones (or the right way) to use them – I’ve been said that at the end, you only use a handful of them – a little like when you switch from imperative to functional programming.

I’ll continue to add examples (for me to train) – if you’re an RxJS expert and find a better way to write my code, please consider to send me a PR.

Resources: