Problem
In my site:
- Users have many Activities
- Each Activity has encoded_polyline data
- I display these encoded_polylines on a map
I want to use IndexedDB (via Dexie) as an in-browser cache so that they don’t need to re-download their full Activity set every time they view their map. I’ve never used IndexedDB before, so I don’t know if I’m doing anything silly or overlooking any edge cases.
Here’s a high-level description of what I think the overall process is:
- Figure out what exists on the server
- Remove anything that is present in IndexedDB but is not present on the server
- Figure out what exists in IndexedDB
- Request only the data missing in IndexedDB
- Store the new data in IndexedDB
- Query all of the data out of IndexedDB
Throughout all of this, we need to be focusing on this user. A person might view many people’s pages, and therefore have a copy of many people’s data in IndexedDB. So the queries to the server and IndexedDB need to be aware of which User ID is being referenced.
Here’s the English Language version of what I decided to do:
- Collect all of this User’s Activty IDs from the server
- Remove anything in IndexedDB that shouldn’t be there (stuff deleted from the site that might still exist in IndexedDB)
- Collect all of this User’s Activty IDs from IndexedDB
- Filter out anything that’s present in IndexedDB and the server
- If there are no new encoded_polylines to retrieve then
putItemsFromIndexeddbOnMap(described below) - If there are new encoded_polylines to retrieve: retrieve those from the server, then store them in IndexedDB, then
putItemsFromIndexeddbOnMap
For putItemsFromIndexeddbOnMap:
- Get all of this user’s encoded_polylines from IndexedDB
- Push that data into an array
- Display that array of data on the map
Here’s the JavaScript code that does what I’ve explained above (with some ERB because this JavaScript is embedded in a Rails view):
var db = new Dexie("db_name");
db.version(1).stores({ activities: "id,user_id" });
db.open();
// get this user's activity IDs from the server
fetch('/users/' + <%= @user.id %> + '/activity_ids.json', { credentials: 'same-origin' }
).then(response => { return response.json() }
).then(activityIdsJson => {
// remove items from IndexedDB for this user that are not in activityIdsJson
// this keeps data that was deleted in the site from sticking around in IndexedDB
db.activities
.where('id')
.noneOf(activityIdsJson)
.and(function(item) { return item.user_id === <%= @user.id %> })
.keys()
.then(removeActivityIds => {
db.activities.bulkDelete(removeActivityIds);
});
// get this user's existing activity IDs out of IndexedDB
db.activities.where({user_id: <%= @user.id %>}).primaryKeys(function(primaryKeys) {
// filter out the activity IDs that are already in IndexedDB
var neededIds = activityIdsJson.filter((id) => !primaryKeys.includes(id));
if(Array.isArray(neededIds) && neededIds.length === 0) {
// we do not need to request any new data so query IndexedDB directly
putItemsFromIndexeddbOnMap();
} else if(Array.isArray(neededIds)) {
if(neededIds.equals(activityIdsJson)) {
// we need all data so do not pass the `only` param
neededIds = [];
}
// get new data (encoded_polylines for display on the map)
fetch('/users/' + <%= @user.id %> + '/encoded_polylines.json?only=' + neededIds, { credentials: 'same-origin' }
).then(response => { return response.json() }
).then(newEncodedPolylinesJson => {
// store the new encoded_polylines in IndexedDB
db.activities.bulkPut(newEncodedPolylinesJson).then(_unused => {
// pull all encoded_polylines out of IndexedDB
putItemsFromIndexeddbOnMap();
});
});
}
});
});
function putItemsFromIndexeddbOnMap() {
var featureCollection = [];
db.activities.where({user_id: <%= @user.id %>}).each(activity => {
featureCollection.push({
type: 'Feature',
geometry: polyline.toGeoJSON(activity['encoded_polyline'])
});
}).then(function() {
// if there are any polylines, add them to the map
if(featureCollection.length > 0) {
if(map.isStyleLoaded()) {
// map has fully loaded so add polylines to the map
addPolylineLayer(featureCollection);
} else {
// map is still loading, so wait for that to complete
map.on('style.load', addPolylineLayer(featureCollection));
}
}
}).catch(error => {
console.error(error.stack || error);
});
}
function addPolylineLayer(data) {
map.addSource('polylineCollection', {
type: 'geojson',
data: {
type: 'FeatureCollection',
features: data
}
});
map.addLayer({
id: 'polylineCollection',
type: 'line',
source: 'polylineCollection'
});
}
Can you review my code for best practices?
Solution
Overall this code looks fine, except I don’t see much for error handling. I see a catch block in the putItemsFromIndexeddbOnMap() function but it either outputs the stack or else the error. What happens if the server responses from the calls to fetch() yield client side errors (e.g. 403, 404, etc) or server-side errors (e.g. 500)? Should the user see a “friendly” message, informing him/her as to what happened?
There are a few simplifications mentioned below. You could also consider using asynchronous functions along with the await operator on promises to remove the .then() blocks.
This code makes decent use of arrow functions, but some of them can be simplified – for instance, after the fetch() call to get activity ids there is this line:
).then(response => { return response.json() }
An arrow function with a single statement doesn’t need curly brackets or a return statement – it can simply be:
).then(response => response.json()
And similarly, there is an identical line after the call to fetch() for encoded polylines – that can also be simplified.
There is also an anonymous function that could be simplified using an arrow function:
.and(function(item) { return item.user_id === <%= @user.id %> })
There are a few anonymous functions that could be simplified to function references
For example:
.then(removeActivityIds => {
db.activities.bulkDelete(removeActivityIds);
});
Should be simplifiable to
.then(db.activities.bulkDelete)
Though you might have to make a bound function with .bind()
And this block:
db.activities.bulkPut(newEncodedPolylinesJson).then(_unused => {
// pull all encoded_polylines out of IndexedDB
putItemsFromIndexeddbOnMap();
});
Can be simplified to
db.activities.bulkPut(newEncodedPolylinesJson)
.then(putItemsFromIndexeddbOnMap); // pull all encoded_polylines out of IndexedDB
On the line below, Array.isArray() seems to be the wrong place to check for an array:
if(Array.isArray(neededIds) && neededIds.length === 0) {
Given that it is returned from:
var neededIds = activityIdsJson.filter((id) => !primaryKeys.includes(id));
And Array.filter() returns “A new array with the elements that pass the test. If no elements pass the test, an empty array will be returned.“1
So perhaps the check should really be on activityIdsJson to see if that is an array.