Packing Flagstones

Washington, USA | 2023 - Present

Using OpenCV and homography to document flagstone geometry, in pursuit of packing them using various methods.

Overview

Dry stone construction can be an inherently difficult task - the material is heavy and its geometry can be highly variable. For millennia, builders have struggled with the challenge of arranging irregularly shaped rocks efficiently. The manual process of selecting, arranging and reshaping these stones requires a great amount of time and energy, such that in the modern-day developed world building with these materials has become increasingly rare. Leveraging cutting edge technology to automate some of this process helps alleviate this, whilst also reducing waste.

Only a year after surviving a very scary accident involving a 5m/15' plunge from a ladder, the client was faced with the daunting task of hauling around and arranging 330 flagstones in their back garden. Before getting involved, only about 10 stones over the course of 9 months had been laid - a feat which had taken the client a total of a day or so after careful selection, arranging, chiseling, rearranging, laying, chiseling, etc. An opportunity was seized to help facilitate this arduous process and as well employ some newly acquired tools gained in the worlds of photogrammetry, OpenCV, grasshopper, physics simulation, and iterative nesting tools.

In Detail

Photogrammetry

Given the often geometrically organic nature of landscaping, the client's penchant for curves, and that some work had already been started, making detailed as-builts was going to be challenging. In lieu of taking a multitude of measurements and attempting to triangulate everything just to achieve a 2D plan, photogrammetry was utilized to generate a digital twin of the site. This would also help to document any nooks or outcroppings of house or yard that may otherwise have been left undiscovered, as well as capture the irregular geometry where stones had already been laid, and ultimately supplement the visuals.

Documentation

As each of these stones weigh roughly somewhere between 5-50kg (~10-100lbs) this was a daunting task and required two people doing two days of heavy lifting. Careful planning and building redundancies into the process were crucial and ensured that information wouldn't be lost. Thus the digital workflows mentioned below had contingencies.

Setup

Calibration

In our first step processing the images, camera calibration aims to remove the radial and tangential distortion of the camera's lens. This is calculated with OpenCV using a series of photos containing a chessboard pattern. The output is a set of parameters that can be fed back into code to process the remaining images.

Homography

Homography, in our case, removes all perspective distortion from the images leaving us with an orthogonal 2D representation of our 3 dimensional stage. Given that the aruco markers are in a fixed location we can use this to determine the size and shape of the stones facing the camera.

Contouring

To isolate the stones from the rest of the scene, the images are run through a series of filters to best differentiate their pixel values from the blue background. In our process, the blue background itself is isolated and the largest outline (the outer edge of the tape) is removed from the array of contours. The outlines are then exported to vector graphics as svg files.

Sorting 1: Brute Force Nesting

The first attempt at packing using the grasshopper component OpenNest which uses a No-Fit Polygon (NFP) approach to iteratively reorient the polygons in relationship to eachother and a bounding geometry. Unfortunately the results left much to be desired as the plug-in prefers a straight edge for the initial alignment and wasn't able to account for the voids in the bounding geometry. It became apparent with this approach that an important first step in form finding would need to be aligning the geometry along the edges first and pack the remaining geometry in amongst the remaining space.

Sorting 2: Physics Simulations

Given these learnings from the first attempt in mind. This iteration tried to use physics-based approach to let the stones pack themselves comfortably amongst one another. Like large grains of sand falling through an unusually shaped hourglass, this approach feeds the rocks into the space with gravity, allowing them to settle amongst each other and the bounding geometry as comfortably as possible. This yielded a much more favorable result.

Sorting 3: Laser Cut Puzzle Time!

After running the physics simulation through several iterations, a laser cut scale model of the site was made to explore and customize the final design.

This gave control and design flexibility back into the hands of the client and would utlimately allow them to execute the project as they see fit by moving small pieces of wood instead of massive stones.

Laying the Stones

Within an hour of the puzzle's creation, real-world progress had already been made.