This isn’t a dropshipping success story, a lucrative exit, or a guide. This is a silly story documenting our failed hardware startup. It’s embarrassing to fail, so we seldom hear these stories, but I’m going to tell you one, here goes.
BobbyTables2 14 hours ago [-]
The SAD algorithm looks neat!
How long does it take? Wouldn’t there be a large range of offsets? Or does limiting the minimum distance help mitigate such?
I suppose there could be corner cases in which an object with regular pattern could be misinterpreted as being far away ?
throwaway219450 13 hours ago [-]
SAD runs in better than real time on a typical desktop CPU, even on a Pi. The author glossed over basically all of the calibration involved with stereo. It’s not quite as simple as taking two cameras and running a matcher.
Because stereo disparity (parallax) scales inversely with distance, conveniently you don’t need to search huge ranges. A point at infinity should have zero disparity in a calibrated stereo system. So normally you pick a depth range and search the corresponding pixels. Also if the cameras are calibrated, you only need to search horizontally so it works out to say 64 operations per pixel, which can be vectorized if the CPU supports it. Other approaches use more stable comparisons like census (is each pixel in the match window brighter than the central one) and hamming distance which is also very efficient.
And yes, stereo is sensitive to repeating texture but absence of texture like a blank wall is usually worse.
hex4def6 12 hours ago [-]
I'm not convinced he's done the calibration, at least in his demo video. It's pretty bad, imo.
I think he's just running it on the raw unrectified images.
For anyone wanting to play with this, OpenCV has a number of tutorials on this using stuff like stereoBM (which also uses SAD if I'm remembering my algos).
The first step is calibration -- intrinsics (basically figuring out the equations that can cancel out the lens distortion) and extrinsics (how far apart are the two cameras in XYZ, tilt, etc).
globalnode 11 hours ago [-]
can this be done manually by setting constants till the images look rectified? last time i played with opencv for this it didnt do a very good job at automatically calculating the k's. im hoping an algo comes out that lets me do disparity without calibration one day :)
dima55 36 minutes ago [-]
Depends on how much accuracy you want in your results. You can do chessboardless calibration and stereo today, but you need lot of data and careful error propagation that nobody does. The mrcal project does does high-precision calibration and streo with boards. I've a boardless method mostly done now, and will run experiments soon. The conclusion might be that you need unreasonable input data to get good results :)
throwaway219450 11 hours ago [-]
You can use structure from motion for this. Walk around and capture some video. It’s the same way that drone photogrammetry works, ie bundle adjustment. It’ll give you the intrinsics at least.
But with a high res sensor, even flex in the PCB will invalidate the calibration. Only takes a few microns for a pixel to be in the wrong place.
numpad0 5 hours ago [-]
I've fed like 1s interval mp4 screenshots from Vive Flow into a Gemini vibecoded Python cv2 script for one of my failed project and that part seemed to work fine, though it was a stereo camera specifically designed and validated for SLAM and also there was a catch that the entirety of the board is clearly visible for a single image to be usable. Maybe try reducing square counts and adjusting illumination if it's not working out.
btw K is scale factor right? If other parameters were good, maybe you can calculate that from sensor dimensions and focal length if scale was only the problem.
michaelt 8 hours ago [-]
While I wouldn't say opencv makes it simple they do provide functions like findChessboardCorners, CharucoDetector, stereoCalibrate - even code to generate printable calibration patterns.
And these days, you can get an LLM to stitch those functions together
choppaface 10 hours ago [-]
lots of “world models” are pretty good at “stereo with zero calibration”
Can also help if you can remove IR filter + add an IR noise projector
rasz 6 hours ago [-]
Tons of problems to take care of, especially if you only want to do horizontal passes.
- sensors have to be mounted solid
- sensors have to be mechanically calibrated in two axes
- coefficients of thermal expansion of all the materials between sensors
Afaik most good stereo systems have camera affixed to a heavy lump of alloy
XZeuwsXJ 12 hours ago [-]
This is cool. Very well done. I wish I needed it for something, I'd take one for sure.
solarSail34 10 hours ago [-]
It would be useful to see Built on a second platform before drawing a general conclusion.
ThrowawayTestr 15 hours ago [-]
Neat! What's the BOM cost?
numpad0 12 hours ago [-]
... why aren't there mentions of stereo baseline distance and clock signal distribution challenges?
Also can't you just buy the PS5 camera? IIRC they switched to regular USB plugs for PS5 with adapters available for PS4 cameras.
lagrange77 9 hours ago [-]
> Also can't you just buy
Do you ask that question on every Open Source / Open Hardware project you encounter?
numpad0 7 hours ago [-]
There's usually a clear use case and a motivation, like you were building an AR passthrough goggles and existing cameras were too big or you didn't want a USB3 camera for it. It's not clear what those were for the author, and the article is rather information sparse relative to the complexity of the project.
alphakappa 1 hours ago [-]
You can assume that the motivation is for the author to build something and share it with like-minded hackers. If he picked something OTS, that wouldn't be a story on HN.
This isn’t a dropshipping success story, a lucrative exit, or a guide. This is a silly story documenting our failed hardware startup. It’s embarrassing to fail, so we seldom hear these stories, but I’m going to tell you one, here goes.
How long does it take? Wouldn’t there be a large range of offsets? Or does limiting the minimum distance help mitigate such?
I suppose there could be corner cases in which an object with regular pattern could be misinterpreted as being far away ?
Because stereo disparity (parallax) scales inversely with distance, conveniently you don’t need to search huge ranges. A point at infinity should have zero disparity in a calibrated stereo system. So normally you pick a depth range and search the corresponding pixels. Also if the cameras are calibrated, you only need to search horizontally so it works out to say 64 operations per pixel, which can be vectorized if the CPU supports it. Other approaches use more stable comparisons like census (is each pixel in the match window brighter than the central one) and hamming distance which is also very efficient.
And yes, stereo is sensitive to repeating texture but absence of texture like a blank wall is usually worse.
I think he's just running it on the raw unrectified images.
For anyone wanting to play with this, OpenCV has a number of tutorials on this using stuff like stereoBM (which also uses SAD if I'm remembering my algos).
The first step is calibration -- intrinsics (basically figuring out the equations that can cancel out the lens distortion) and extrinsics (how far apart are the two cameras in XYZ, tilt, etc).
But with a high res sensor, even flex in the PCB will invalidate the calibration. Only takes a few microns for a pixel to be in the wrong place.
btw K is scale factor right? If other parameters were good, maybe you can calculate that from sensor dimensions and focal length if scale was only the problem.
And these days, you can get an LLM to stitch those functions together
but if you want to calibrate, try printing out a board https://calib.io/pages/camera-calibration-pattern-generator
- sensors have to be mounted solid
- sensors have to be mechanically calibrated in two axes
- coefficients of thermal expansion of all the materials between sensors
Afaik most good stereo systems have camera affixed to a heavy lump of alloy
Also can't you just buy the PS5 camera? IIRC they switched to regular USB plugs for PS5 with adapters available for PS4 cameras.
Do you ask that question on every Open Source / Open Hardware project you encounter?