D3 and React Resources

There are a variety of ways to have D3 and React interact since they both modify the DOM. Generally the recommended approach is “use useRef to access a React-rendered DOM element, and then put the D3 logic in a useEffect“.

useEffect(() => {
   if (svgRef.current) {
      const nodeJoin = d3.select(svgRef.current)
                         .selectAll(".node")
                         .data(data.nodes);

      nodeJoin.exit().remove();
      nodeJoin.enter().append("circle")...

      // Update data
      nodeJoin.attr("cx", d => d.x).attr("cy", d => d.y);
   }
}, [data, svgRef])

Blog Posts

https://wattenberger.com/blog/react-and-d3

Libraries

https://airbnb.io/visx/

Youtube Videos

Github Repositories

https://github.com/TheeMattOliver/d3-react-census-us-population-bar

Arrowhead SVG scaling

When using SVG definitions to add arrowheads to your line in D3, use markerUnits and set a stroke-width attribute if you don’t want the arrowhead to scale with the line.

Static arrowhead:

svg.append('defs').append('marker')
        .attr("id", "arrowhead")
        .attr("markerUnits","userSpaceOnUse")
        .attr("viewBox", "0 -5 10 10")
        .attr("refX",32) 
        .attr("refY", -1)
        .attr("markerWidth", 12)
        .attr("markerHeight", 12)
        .attr("orient", "auto")
        .attr("stroke-width",2)
        .append("path")
        .attr("d", "M0,-5L10,0L0,5")
        .attr('fill','rgb(104, 104, 104)');

Scaling Arrowhead definition

svg.append('defs').append('marker')
        .attr("id",'arrowhead')
        .attr('viewBox','-0 -5 10 10') 
        .attr('refX', 32)
        .attr('refY', -1)
        .attr('orient','auto')
        .attr('markerWidth',12)
        .attr('markerHeight',12)
        .attr('xoverflow','visible')
        .append('svg:path')
        .attr('d', 'M 0,-5 L 10 ,0 L 0,5')
        .attr('fill', 'rgb(104, 104, 104)')
 

Thanks to the D3 slack and Eric S for these tips

Django: Request object has no attribute “accepted_renderer”

If you get the error Request object has no attribute “accepted_renderer” or WSGIRequest object has no attribute “accepted_renderer” when using Django Rest Framework and attempting to add a custom BaseRenderer, then there are a few issues that may be causing this. Two of the most prevalent are:

1. Missing pyyaml dependency
   
   https://github.com/encode/django-rest-framework/issues/6300
   
   When attempting to use the rest_framework.schemas get_schema_view, Django Rest Framework requires pyyaml
   
   
   Solution #1:
   
   Install pyyaml
   
   pip install pyyaml
   
   Solution #2:
   
   Use a version of Django Rest Framework that doesn’t require pyyaml for schema
   
   pip install djangorestframework==3.8.0
   
   
2. Invalid renderers settings
   

When setting renderer_classes it’s important to use the api_settings from rest_framework instead of the DEFAULT_RENDERER_CLASSES from your REST_FRAMEWORK Django settings.

from rest_framework.settings import api_settings

renderer_classes = tuple(api_settings.DEFAULT_RENDERER_CLASSES) + (CSVRenderer,)

https://www.django-rest-framework.org/api-guide/settings/

Python string matching and regular expressions

Python is very unintuitive compared to other languages for regular expression and string matching. . You have to import a separate library to get access to the functionality:

import re

Then the python man page is really confusing.  It starts talking about compiling regular expressions.  That may be something that you want to do when you’re doing a thousand searches, but mostly you just want to see if a string matches another string once.  The JavaScript way of doing this is just STRING.match(STRING), but the python version is re.match(REGEX, STRING).  Not only that, but there’s a HIDDEN CAVEAT in the python version – re.match only matches from the START of the string.  I don’t know why this is the case, but the method you want to use is re.search(REGEX, STRING).  This returns a SRE_Match object… ??  Apparently you use .group() on that object to get the actual match back… but even if you use a REGEX that should match multiple things in the string, it’ll only give you the first one…  Why is this so difficult again??

If you want to avoid the re package AND you have a string that you want to find a match only at the BEGINNING or only at the END, then you can use STRING.startswith(STRING) or STRING.endswith(STRING) to return a True or False Boolean.  Note that this doesn’t actually give you the match… it just tells you that it exists or not…

If you need to find multiple matches and you’re already importing re then you can use the findall method – re.findall(REGEX, STRING) , which strangely returns a list (of course)… though that’s better than returning an SRE_Match object I guess.

 

Get objects that match a specific attribute/property from a list

You have a list of objects where you want to find the object(s) in the list that have a particular value for a property or attribute.

fruits = [{'fruit': 'apple' }, {'fruit': 'pear'}, {'fruit': 'banana'}, {'fruit': 'grape'}]

Python:

list(filter(lambda item: item['fruit'] == 'pear', fruits))

[item for item in fruits if item['fruit'] == 'pear']

Javascript:

fruits.filter(item => item['fruit'] === 'pear')

 

 

d3 mouse on click event target parentNode

To select the parent of an on click event:

.on('click', (data) => {
   d3.select(d3.event.target.parentNode).attr('class', 'new-class')
}

or

on('click', function(){
    d3.select(this.parentNode).classed('new-class', true)
 })

https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Functions/Arrow_functions#No_separate_this

With the fat arrow version, this is set to undefined so you have to use d3.select to get the element again.

With the function version, this is set to the element.

 

NodeBots

Hello world of robotics is turn on an LED.  Thinking about things in bytes with Node.js

johnny-five lets you think in terms of servos, sensors, and LEDS.

IO Classes: johnny-five running everywhere on lots of different hardware

Web to Nodebots…. Node.js with HTTP/REST/johnny-five, EventSource, Socket.io, MQTT

 

 

Ten Rules for Coding with D3

This is an interesting set of rules. Good to know that style overrides attr and the cautions on types are very relevant for parsing numerical data.

Northlandia

Anyone familiar with JavaScript who has tried their hand at D3 knows that coding in it is a little, well, different. For instance, take this snippet of code included in the dummy example I wrote for the UW-Madison Geography 575 lab my students were just assigned:

var provinces = map.selectAll(".provinces")
    .data(topojson.feature(europe, europe.objects.FranceProvinces).features)
    .enter()
    .append("g")
    .attr("class", "provinces")
    .append("path")
    .attr("class", function(d) { return d.properties.adm1_code })
    .attr("d", path)
    .style("fill", function(d) {
        return choropleth(d, colorize);
    })
    .on("mouseover", highlight)
    .on("mouseout", dehighlight)
    .on("mousemove", moveLabel)
    .append("desc")
    .text(function(d) {
        return choropleth(d, colorize);
    });

Now, someone who is familiar with jQuery or Leaflet will probably recognize the method chaining, and some of the methods may even look familiar. But what’s really going on here is somewhat more complex than the syntax lets on. This fall, I’ve had to put a lot of attention into figuring out how to teach this powerful data visualization library to Cartography majors, many…

View original post 2,451 more words

d3 General Update Pattern and Attributes

The d3 pattern to add data to a graph is the general update pattern:

(selection).data().enter().append(new-nodes).attr(something).merge().attr(something)

You select something in the graph (typically an element type or a CSS class), you tell d3 what data you’re going to use with that element, you enter that selection, you append new nodes to the graph, you add some attributes to those new nodes, you grab all of the old nodes with merge, and then you set attributes on all of the nodes to finish.

https://bl.ocks.org/cmgiven/32d4c53f19aea6e528faf10bfe4f3da9

The good practice is to set fixed attributes on enter and variable attributes on update.  If the attribute might change, then you need to set it on update or transition.  This updates our pattern to:

(selection).data().enter().append(new-nodes).attr(something static).merge().attr(something variable)

 

JointJS with npm, Angular, Gulp, and SystemJS

JointJS requires Backbone, Lodash, and JQuery. Backbone requires underscore. Lodash is a replacement for underscore. So in order to use Backbone with Lodash for JointJS you need to configure SystemJS to load Lodash when asked for Underscore by Backbone. This is pretty much the same as their documentation on RequireJS at http://resources.jointjs.com/tutorial/requirejs

I have this set up with npm to manage the JavaScript dependencies, Gulp to copy the files around and SystemJS to load them.


npm install --save jointjs
npm install jquery@3.1.1 --save
npm install lodash@3.10.1 --save
npm install backbone@1.3.3 --save
npm install -D @types/jquery
npm install -D @types/jointjs
npm install -D @types/lodash
npm install -D @types/backbone


gulpfile.js

/* eslint-env node, es6 */
/* eslint no-console: off */

////////////////////////////////
// Setup //
////////////////////////////////

// Plugins
var gulp = require('gulp'),
pjson = require('./package.json'),
gutil = require('gulp-util'),
sass = require('gulp-sass'),
autoprefixer = require('gulp-autoprefixer'),
cssnano = require('gulp-cssnano'),
rename = require('gulp-rename'),
del = require('del'),
plumber = require('gulp-plumber'),
pixrem = require('gulp-pixrem'),
uglify = require('gulp-uglify'),
imagemin = require('gulp-imagemin'),
exec = require('child_process').exec,
runSequence = require('run-sequence'),
browserSync = require('browser-sync').create(),
reload = browserSync.reload,
ts = require('gulp-typescript'),
sourcemaps = require('gulp-sourcemaps'),
merge = require('merge-stream');

// Relative paths function
var pathsConfig = function(appName) {
this.app = './' + (appName || pjson.name);
this.static = this.app + '/static';
this.frontend = this.app + '/frontend';

return {
app: this.app,
templates: this.app + '/templates',
static: this.static,
css: this.static + '/css',
sass: this.static + '/sass',
fonts: this.static + '/fonts',
images: this.static + '/images',
js: this.static + '/js',
frontend: this.frontend,
build: this.frontend + '/build',
node: './node_modules',
};
};

var paths = pathsConfig();
var tsProject = ts.createProject('tsconfig.json');
var moduleDirs = {
'jointjs/dist/*.{js,css,png}': 'libs/jointjs',
'lodash/*.js': 'libs/lodash',
};

var modules = {
'core-js/client/shim.min.js': 'libs/core.js',
'zone.js/dist/zone.js': 'libs/zone.js',
'systemjs/dist/system.js': 'libs/system.js',
'@angular/core/bundles/core.umd.js': 'libs/@angular/core/bundles/core.umd.js',
'@angular/common/bundles/common.umd.js': 'libs/@angular/common/bundles/common.umd.js',
'@angular/compiler/bundles/compiler.umd.js': 'libs/@angular/compiler/bundles/compiler.umd.js',
'@angular/platform-browser/bundles/platform-browser.umd.js': 'libs/@angular/platform-browser/bundles/platform-browser.umd.js',
'@angular/platform-browser-dynamic/bundles/platform-browser-dynamic.umd.js': 'libs/@angular/platform-browser-dynamic/bundles/platform-browser-dynamic.umd.js',
'@angular/http/bundles/http.umd.js': 'libs/@angular/http/bundles/http.umd.js',
'@angular/http/bundles/http-testing.umd.js': 'libs/@angular/http/bundles/http-testing.umd.js',
'@angular/router/bundles/router.umd.js': 'libs/@angular/router/bundles/router.umd.js',
'@angular/forms/bundles/forms.umd.js': 'libs/@angular/forms/bundles/forms.umd.js',
'jquery/dist/jquery.min.js': 'libs/jquery/dist/jquery.min.js',
'vex-js/dist/css/vex.css': 'css/vex.css',
'vex-js/dist/css/vex-theme-plain.css': 'css/vex-theme-plain.css',
'backbone/backbone-min.js': 'libs/backbone-min.js'
};

////////////////////////////////
// Tasks //
////////////////////////////////

gulp.task('clean', function() {
return del.sync(paths.build);
});

gulp.task('collectDirs', ['clean'], function() {
var streams = [];
Object.keys(moduleDirs).forEach(function(path) {
streams.push(
gulp.src(path, {'cwd': paths.node})
.pipe(gulp.dest(moduleDirs[path], {'cwd': paths.build}))
);
});
return merge(streams);
});

gulp.task('collectModules', ['clean'], function() {
var streams = [];
Object.keys(modules).forEach(function(path) {
streams.push(
gulp.src(path, {'cwd': paths.node})
.pipe(rename(modules[path]))
.pipe(gulp.dest(paths.build))
);
});
return merge(streams);
});

gulp.task('collectAppResources', ['clean'], function() {
return gulp.src(['**/*.html', '**/*.css'], {'cwd': paths.frontend})
.pipe(gulp.dest(paths.build + '/app'))
});

// Collects libs from node_module
gulp.task('collect', ['collectDirs', 'collectModules', 'collectAppResources']);

// Compile TypeScript Files
gulp.task('app', ['collect'], function() {
return gulp.src(paths.frontend + '/**/*.ts')
.pipe(sourcemaps.init())
.pipe(tsProject())
.pipe(sourcemaps.write())
.pipe(gulp.dest(paths.build + '/app'));
});
// Javascript minification
gulp.task('scripts', function() {
return gulp.src(paths.js + '/project.js')
.pipe(plumber()) // Checks for errors
.pipe(uglify()) // Minifies the js
.pipe(rename({suffix: '.min'}))
.pipe(gulp.dest(paths.js));
});

// Image compression
gulp.task('imgCompression', function() {
return gulp.src(paths.images + '/*')
.pipe(imagemin()) // Compresses PNG, JPEG, GIF and SVG images
.pipe(gulp.dest(paths.images));
});
// Task to run ./manage.py/collectstatic
gulp.task('collectStatic', ['app'], function() {
exec('python manage.py collectstatic --noinput', function(err, stdout, stderr) {
console.log('Collecting static files');
console.log(stdout);
console.log(stderr);
});
});

// Default task
gulp.task('default', function() {
return runSequence(['app']);
});


systemjs configuration:

(function () {
System.config({
// map tells the System loader where to look for things
map: {
'app': '/static/app',

// angular bundles
'@angular/core': '/static/libs/@angular/core/bundles/core.umd.js',
'@angular/common': '/static/libs/@angular/common/bundles/common.umd.js',
'@angular/compiler': '/static/libs/@angular/compiler/bundles/compiler.umd.js',
'@angular/platform-browser': '/static/libs/@angular/platform-browser/bundles/platform-browser.umd.js',
'@angular/platform-browser-dynamic': '/static/libs/@angular/platform-browser-dynamic/bundles/platform-browser-dynamic.umd.js',
'@angular/http': '/static/libs/@angular/http/bundles/http.umd.js',
'@angular/http/testing': '/static/libs/@angular/http/bundles/http-testing.umd.js',
'@angular/router': '/static/libs/@angular/router/bundles/router.umd.js',
'@angular/forms': '/static/libs/@angular/forms/bundles/forms.umd.js',

// other
'rxjs': '/static/libs/rxjs',
'jquery': '/static/libs/jquery/dist/jquery.min.js',
'lodash': '/static/libs/lodash/index.js',
'underscore': '/static/libs/lodash/index.js',
'*': {
'underscore': '/static/libs/lodash/index.js',
},
'backbone': '/static/libs/backbone-min.js',
'jointjs': '/static/libs/jointjs/joint.js',
},
// packages tells the System loader how to load when no filename and/or no extension
packages: {
app: {
main: './main.js',
defaultExtension: 'js'
},
rxjs: {
defaultExtension: 'js'
},
},
meta: {
'jointjs': {
exports: 'joint',
deps: ['jquery', 'lodash', 'underscore', 'backbone']
},
'backbone': {
deps: ['underscore'],
},
}
});
})();