Skip to content
days.

Latest 🗞️

Saturday, August 22

Git Commits

dash commits 3 ohnsh/dash

  • 9:30 pm — John Sherrell <dev@ohn.sh>

    vod: fixed a couple of silly bugs in video.sh. `count` is now local (but initialized from the environment) so that it's re-computed after the script sleeps in watch mode. And now actually using `status` after saving it, instead of incorrectly repeating `$?`.

  • 4:08 pm — John Sherrell <dev@ohn.sh>

    dashd: migrated sensor db to use cleaner column names, added location for second homepod (bedroom).

  • 1:16 am — John Sherrell <dev@ohn.sh>

    dashd: now logging sensor readings to an sqlite db.

YouTube

Friday, August 21

Git Commits

itsa-me commits 1 ohnsh/itsa-me

  • 3:04 am — John Sherrell <dev@ohn.sh>

    Links that could go somewhere.

dash commits 2 ohnsh/dash

  • 2:15 am — John Sherrell <dev@ohn.sh>

    dashd: new route, POST /mx/sensor, to which sensor data can be sent by automation. Data is currently logged to a jsonl file, but soon it will be an sqlite database. Auth is checked against a static token passed in the environment.

  • 12:14 am — John Sherrell <dev@ohn.sh>

    Testing recharts.

Thursday, August 20

First pass at an interactive picture-in-picture video component

Much of the video that winds up on dash.ohn.sh is recorded in a very particular format (in OBS Studio), consisting of four 1080p tiles, each representing an “angle” of the same scene. Two of those angles are screen captures, which are difficult to read when limited to one quarter of a video player. Luckily, the web platform provides a bunch of tools to process video and create custom video players. The above demo shows a first pass at a solution, implemented as an interactive React component.

Concept

The basic idea is to display one quadrant at a time in a main view that occupies most of the video player, with the other three quadrants shown picture-in-picture style along one edge. Clicking a picture-in-picture view swaps it with the main view. Clicking the main view toggles all of the picture-in-picture views on/off as a group, which is important because they do obscure part of the main view when enabled.

View full post…

Git Commits

dash commits 8 ohnsh/dash

  • 4:21 pm — John Sherrell <dev@ohn.sh>

    @dash/next: more accessibility improvements to CropOverlay. The static data structure that defines the crop regions now includes descriptive text that is ised in `aria-label` attributes.

  • 3:19 pm — John Sherrell <dev@ohn.sh>

    @dash/next: improve accessibility in CropOverlay by wrapping interactive <canvas> elements in <button>s that are given the onClick actions.

  • 12:59 pm — John Sherrell <dev@ohn.sh>

    @dash/next: make focus outline styles work in Safari, too, it an annoyingly fragile way. The fixes currently only apply to the VODPlayer views; the sidebar is next.

  • 12:41 pm — John Sherrell <dev@ohn.sh>

    @dash/next: improve styling of links and buttons in VODPlayer views by creating room for browser-default focus outlines inside scroll containers, where they are otherwise clipped. Update buttons underneath VODPlayer to simply brighten on hover rather than use outline styles that could be confused with focus.

  • 10:56 am — John Sherrell <dev@ohn.sh>

    @dash/next: CropOverlay: make pipEnabled true by default (when overlay is activated). Add `cursor: pointer` to "See more" button in sidebar.

  • 1:55 am — John Sherrell <dev@ohn.sh>

    @dash/next: fixed another layout bug, this time for landscape mobile viewports. At some point, I changed `overflow: hidden` to `overflow: clip` on the <video> wrapper element, which removed the flexbox min-size reset associated with scroll containers. I'm now setting `min-height: 0` directly.

  • 1:15 am — John Sherrell <dev@ohn.sh>

    @dash/next: better CSS handling of VODPlayer when no source selected. Uses `vh` unit for both width and max-height in order to limit the size while maintaining a reasonable aspect ratio. (Unfortunately, `aspect-ratio` itself doesn't seem to constraing width when `max-height` is reached.)

  • 1:06 am — John Sherrell <dev@ohn.sh>

    @dash/next: fixed min-height bug that caused VODPlayer to overflow the viewport in portrait mode at mobile sizes.

days commits 4 ohnsh/days

  • 10:34 pm — John Sherrell <dev@ohn.sh>

    Tag markdown code blocks as typescript for syntax highlighting.

  • 8:52 pm — John Sherrell <dev@ohn.sh>

    Document typos, vale, and cspell usage (spellcheckers / prose linters).

  • 8:17 pm — John Sherrell <dev@ohn.sh>

    CropOverlay demo video was too large for `wrangler deploy`; moved to a new R2 bucket.

  • 7:55 pm — John Sherrell <dev@ohn.sh>

    New post about CropOverlay component in @dash/next, along with demo video. Ran `typos` on project and fixed a few typos.

Wednesday, August 19

Git Commits

dash commits 7 ohnsh/dash

  • 11:54 pm — John Sherrell <dev@ohn.sh>

    @dash/next: reasonably functional picture-in-picture view now implemented in CropOverlay component. The option to activate it appears when viewing the quad-tile 4K videos I record in OBS. When activated, a quadrant from the underlying video is cropped out and overlaid on top, covering most of the video but leaving room for the controls underneath. Clicking the overlay brings up picture-in-picture views of the other three quadrants. Clicking one of the PiP views swaps it into the main canvas. Clicking the main canvas switches off picture-in-picture, and clicking the button below the video player toggles the overlay as a whole.

  • 9:35 pm — John Sherrell <dev@ohn.sh>

    @dash/next: quick fix to VODPlayer segments list; add `padding: 1px` to make room for outline around active segment.

  • 9:21 pm — John Sherrell <dev@ohn.sh>

    @dash/next: VODPlayer has a new div.footer that consolidates under-video content. It is a grid container with three columns so that the crop button, when present, is always left-aligned, and the segment timestamps, when present, are always centered.

  • 7:50 pm — John Sherrell <dev@ohn.sh>

    @dash/next: new CropOverlay component that renders a cropped quadrant of an underlying <video> in a <canvas>. Currently, it can be activated by a button that's only rendered underneath 4K videos. Clicking the canvas causes the other three quadrants in picture-in-picture views on the same canvas. This should be improved by using a different <canvas> for each view, and making them interactive so that they can be swapped with the main view on click/tap.

  • 5:03 pm — John Sherrell <dev@ohn.sh>

    @dash/next: add aria-expanded and aria-controls attributes to InventoryList "See more" button.

  • 4:24 pm — John Sherrell <dev@ohn.sh>

    @dash/next: make sidebar archive menu expandable. By default, it displays the 8 most-recent days and a "See more" button.

  • 12:25 am — John Sherrell <dev@ohn.sh>

    next: simplified layout by not limiting <article> to viewport height. The main player is now contstrained by explicit rules on itself, whereas before it was constrained by the container and the flex-basis of the thumbstrip section beneath.

days commits 1 ohnsh/days

  • 3:48 pm — John Sherrell <dev@ohn.sh>

    Finished 8/18 post about Silero VAD.

Tuesday, August 18

dash.ohn.sh — Automated content filtering powered by Silero VAD

For a new project, I have a video processing pipeline that prepares raw camera footage for publishing, extracts metadata, and syncs everything to a Cloudflare R2 bucket. The newest feature is voice activity detection with Silero VAD, which I’m running on the CPU via onnxruntime-node. Here it is taking around 20 seconds to analyze several hours of footage on my Mac Mini.

The model takes a tensor representing 512 samples of 16-kHz, single-channel audio (roughly 30 ms) and returns a score indicating the probability that the window contains speech (“voice activity”). In the pipeline, audio is extracted from source recordings and normalized with ffmpeg, then fed through the model piece by piece. The raw scores are filtered through a state machine (during a speech segment, a given sample is more likely to register as speech) to produce a list of timestamps representing segments of the video that contain speech.

View full post…

Git Commits

dash commits 3 ohnsh/dash

  • 2:30 pm — John Sherrell <dev@ohn.sh>

    next,vod: add optional `tags` array to VODVideo schema, surface videos with the property on the root route along with those with detected speech.

  • 1:21 pm — John Sherrell <dev@ohn.sh>

    next: change format and style of VODPlayer timestamp. It's now absolutely positioned within the video frame, at bottom right.

  • 10:27 am — John Sherrell <dev@ohn.sh>

    next: handle wyze filenames (HH.mp4) when computing timestamps by falling back to a function that takes a date and filename. Pass the current inventory record to the thumbstrip component rendering it so that it can be filtered on the client.

days commits 1 ohnsh/days

  • 4:25 pm — John Sherrell <dev@ohn.sh>

    New post about Silero VAD, bump Astro version.

Monday, August 17

Git Commits

dash commits 5 ohnsh/dash

  • 10:30 pm — John Sherrell <dev@ohn.sh>

    dashd: add wyze2 configs to env.sh and compose.yml, plus the dashd snapshot route handler.

  • 10:02 pm — John Sherrell <dev@ohn.sh>

    next: add wyze2, remove desk and quad from live route for now.

  • 5:07 pm — John Sherrell <dev@ohn.sh>

    next: useCrop kind of works but it prevents native seek controls from working. The ultimate answer will be rendering custom controls on a <canvas> or <video> with srcObject fed from a canvas. Leaving out of VODPlayer for now.

  • 9:39 am — John Sherrell <dev@ohn.sh>

    next: useCrop is rough but functional. Keeping in a separate branch while I decide on a UI.

  • 12:30 am — John Sherrell <dev@ohn.sh>

    next: first pass at a useCrop custom hook that crops a rectangle out of a hidden source <video> and renders it to an on-screen <video> via video.srcObject and canvas.captureStream(). (It doesn't work.)

dev commits 1 ohnsh/dev

  • 9:36 pm — John Sherrell <dev@ohn.sh>

    cam: add wyze2 config to compose.yml extra_hosts and mediamtx paths.

Sunday, August 16

Git Commits

dash commits 9 ohnsh/dash

  • 10:16 pm — John Sherrell <dev@ohn.sh>

    next: extracted a VODView component that implements the main view at both / and /[date], (/ filters by voice activity and /[date] filters by date). The tsToString helper now takes only two paramters and passes `undefined` to `toLocaleString` to use the default (user) locale for date formatting (although not the timezone).

  • 8:48 pm — John Sherrell <dev@ohn.sh>

    vod,next: raise minimum confidence threshold for voice detection. To fix existing data and allow for greater front-end flexibility, start computing speechTotal for a given minimum confidence instead of just using the value computed at detection time. Rename `speechThreshold` property to `minConfidence`, update zod schema to handle both names.

  • 7:52 pm — John Sherrell <dev@ohn.sh>

    next: add star icon with accessible text description to list of voice segments, to match the icon in the thumbnails.

  • 7:29 pm — John Sherrell <dev@ohn.sh>

    Deployment oops. Next package now named @dash/next.

  • 7:13 pm — John Sherrell <dev@ohn.sh>

    next: add voice segment interface to the vod/archive (/[date]) route. The route now mirrors the home route with a flex-based layout and passing a Promise<DashVideo> to <VODPlayer>. Also add star indicator to thumbnails for videos with detected voice segments.

  • 5:19 pm — John Sherrell <dev@ohn.sh>

    Rename packages using org name @dash scoped to this monorepo.

  • 3:14 am — John Sherrell <dev@ohn.sh>

    dash-next: player skeleton fills content area when <video> is absent.

  • 2:47 am — John Sherrell <dev@ohn.sh>

    voice-detect: don't pad segments to below zero or above duration.

  • 2:31 am — John Sherrell <dev@ohn.sh>

    dash-next: make home route layout fluid with the video occupying the content area while maintaining aspect ratio and room for one thumbnail strip at the bottom of the viewport. Use react state and 'timechange' event to highlight the active voice segment if there is one. Also handle negative numbers in segment data.

dev commits 2 ohnsh/dev

  • 4:13 pm — John Sherrell <dev@ohn.sh>

    cam: update cam.sh style to us ALL_CAPS for global script variables, add comments.

  • 1:44 am — John Sherrell <dev@ohn.sh>

    cam: simpler notification mechanism that works without fifo and monitor script. Also organize global variable initialization into a function.

Saturday, August 15

Git Commits

dash commits 2 ohnsh/dash

  • 9:49 pm — John Sherrell <dev@ohn.sh>

    dash-next: helper scripts in package.json for using db dev server, turn home route into a flex layout that respects the viewport height, similar to the way /live has been for a few days.

  • 1:46 pm — John Sherrell <dev@ohn.sh>

    dash-next: rudimentary interface to skip between voice segments on the home route.

Friday, August 14

Git Commits

dash commits 6 ohnsh/dash

  • 11:24 pm — John Sherrell <dev@ohn.sh>

    dash-next: root route now shows only video with speech, along with a (very rough) interface to skip to each speech segment.

  • 8:12 pm — John Sherrell <dev@ohn.sh>

    dash-next: Add library functions for converting between short and long keys for URL query parameters, and for converting between keys and video sources.

  • 6:34 pm — John Sherrell <dev@ohn.sh>

    vod: extracted a helper to compute total speech duration from a VoiceResult, necessary because sometimes the `speechTotal` property is absent. util.ts is a package export.

  • 6:31 pm — John Sherrell <dev@ohn.sh>

    dash-next: new DashVideo type, derived from VODVideo with helpful computed properties. Refactor ThumbStrip to accept a promise that resolves to DashVideo[] instead of implementing its own inventory fetching/parsing logic.

  • 2:41 pm — John Sherrell <dev@ohn.sh>

    dash-next,dash-vod: big refactor to consolidate schemas and keep imports clean. If my vod processing code is anywhere in the import path from Next, it will crash the build. Schemas are now in vod/lib/schema.ts, which doesn't import any code that might cause build issues for depending projects.

  • 12:21 am — John Sherrell <dev@ohn.sh>

    dash-next: front end now using drizzle and the migrated database.

dev commits 1 ohnsh/dev

  • 8:15 pm — John Sherrell <dev@ohn.sh>

    cam: new wrapper function to supervise recordings and restart them in case of a recoverable error. For now, the only recoverable error is when the date changes mid-run and ffmpeg starts trying to save to a folder that doesn't exist (since the path is computed from the date).

Thursday, August 13

Git Commits

dash commits 13 ohnsh/dash

  • 9:47 pm — John Sherrell <dev@ohn.sh>

    voice-detect,vod: log in consistent format.

  • 8:57 pm — John Sherrell <dev@ohn.sh>

    vod: tweak path handling in video.sh, remove leading _ in camera dir names when computing r2 path.

  • 7:53 pm — John Sherrell <dev@ohn.sh>

    vod: crucial to await db insert before letting script exit 😆

  • 7:22 pm — John Sherrell <dev@ohn.sh>

    vod: upsert database entries so that speech total is updated as the file progresses.

  • 7:01 pm — John Sherrell <dev@ohn.sh>

    vod: now using drizzle, along with a slightly more serious table. There's a new column for total speech duration (in seconds), along with supporting code in video.ts.

  • 3:39 pm — John Sherrell <dev@ohn.sh>

    vod: added reset subcommand to video.sh that moves processed files not in inventory.json into an _error_asset subdirectory for processing. (Otherwise the files will be synced to R2 but completely ignored.) Also improved error handling to use specific _error_{error_type} subdirectory names depending on the error.

  • 2:32 pm — John Sherrell <dev@ohn.sh>

    Updated workspaces.

  • 2:29 pm — John Sherrell <dev@ohn.sh>

    dash-next: add drizzle orm.

  • 2:28 pm — John Sherrell <dev@ohn.sh>

    dash-vod: rename from mkoverlay

  • 11:58 am — John Sherrell <dev@ohn.sh>

    mkoverlay: Add try/catch around ffprobe audio stream detection; return false on error. voice-detect: Log stats about speech segments after inference runs.

  • 8:31 am — John Sherrell <dev@ohn.sh>

    mkoverlay: dynamically import() voice-detect and gracefully recover from errors so the pipeline (minus inference) will run on Alpine (onnxruntime-node requires a glibc system). The ultimate solution will be to run the typescript portion of the pipeline in a debian-based docker container. Also remove -xerror from ffmpeg args for now. Unfortunately, lots of video from thingino and mediaMTX has out-of-order timestamp issues, which ffmpeg can eaily recover from. For now, my manual file size check should filter out most of the truly worthless files. If I leave -xerror, I will lose a lot of footage that doesn't need to be lost.

  • 7:35 am — John Sherrell <dev@ohn.sh>

    mkoverlay,voice-detect: Add new voice-detect package that uses Silero VAD, via TypeScript and onnxruntime-node, to detect segments of voice activity in audio or video files. Also updated mkoverlay batch video pipeline to use the new package and include the segment information in inventory.json

  • 6:25 am — John Sherrell <dev@ohn.sh>

    mkoverlay: fixed early loop continuation bug in video.sh on error.

dev commits 2 ohnsh/dev

  • 10:15 pm — John Sherrell <dev@ohn.sh>

    cam: dust off cam.sh to take over recording duties again. MediaMTX cannot seem to reliably record thingino streams, whereas ffmpeg complains about them but produces rock solid output.

  • 8:58 pm — John Sherrell <dev@ohn.sh>

    sh: stop sending files to _capture_2x2, let them go to 'quad' based on the way OBS now saves recordings.