Rendered at 21:18:57 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
longislandguido 27 minutes ago [-]
On the surface this seems like a terrible idea:
FFmpeg is mountains of extremely complex C code whose entire job is processing untrusted inputs.
Choosing to make such code network-enabled if you can't trust your inputs, I would recommend to sandbox if at all possible. Otherwise you are asking for trouble.
steelbrain 23 minutes ago [-]
Thank you for your comment!
The usecase for something like this is when you control both sides, server & client. There is some basic HMAC auth built into each request.
> I would recommend to sandbox if at all possible.
Since the server is a standard binary that doesn't need any special permissions, you could create the most locked down user in your server that only has access to a limit set of files and the GPUs and it'll work just fine. This is encouraged.
steelbrain 3 hours ago [-]
Hi HN!
I'm excited to show case an update to a personal project of mine. Its called ffmpeg-over-ip and it allows you connect to remote ffmpeg servers, so if you have one machine with a GPU (could be your windows gaming laptop, gaming PC, a macbook?) and a machine (or VM, docker container etc) without a GPU, you could use the remote GPU to do GPU-accelerated video conversion.
The way it works is pretty neat, there are two components, a server and a client.
- The server (has the GPU) comes with a patched up ffmpeg and listens on a specified port
- The client (without the GPU) connects to the server, takes the file IO requests from the server and runs them locally.
ffmpeg doesn't know that its not dealing with a local filesystem, so this approach works with multiple inputs or outputs like HLS, and is perfect for home media servers like Plex or Jellyfin or Emby.
One server can offer up its GPU to as many clients as needed, all clients offer up their own filesystems for their requests, the server also comes with a static build of ffmpeg bundled (built from jellyfin-ffmpeg scripts) so all you have to do is create a config file, set a password and you're good to go!
It's been about a year and half since this was last submitted (https://news.ycombinator.com/item?id=41743780). The feedback at the time was around the difficulty of sharing a filesystem between the machines so that should no longer be a problem.
This has been really useful in my local setup, I hope you find it useful. If you have any further questions, the website has some FAQs (linked in github repo), or you could post them here and I'll answer them for you!
Thanks!
bsimpson 18 minutes ago [-]
Nice to see you here!
Was really impressed by your work on Pundle. (It was an amazingly fast HMR dev environment - much like Vite today.) Felt like I was the only one using it, but it was hard to walk away from instant updates.
steelbrain 11 minutes ago [-]
Thanks for putting a smile on my face! I am glad you liked it! :)
I built this macOS app that allows you to use any off the shelf wifi camera as your webcam with Zoom, Microsoft Teams, etc. It has lower latency than OBS, VLC etc based on my testing, its Swift-Native and pretty lightweight.
It was built mainly for my own team so they don't have to run long wires of USB cameras or pay a lot of money for a "wireless webcam". I hope you find it useful!
throwaway81523 56 minutes ago [-]
Maybe you can submit a patch to ffmpeg.org.
steelbrain 52 minutes ago [-]
I've considered it, thanks for the nudge. Since the patches are quite specific to my usecase:
I am not sure they'll be accepted for upstreaming, but in exploring the options, I noticed ffmpeg has sftp:// transport support and there were some bugs surrounding that. I do intend to publish some patches for those.
tonymet 2 hours ago [-]
cool idea. can you elaborate on IO and how the ffmpeg-server reads blocks from the client? that would seem to be a big blocker
steelbrain 2 hours ago [-]
> cool idea. can you elaborate on IO and how the ffmpeg-server reads blocks from the client? that would seem to be a big blocker
ffmpeg-server runs a patched version of ffmpeg locally, ffmpeg requests to read some chunks (ie give me video.mp4) through our patched filesystem (https://github.com/steelbrain/ffmpeg-over-ip/blob/main/fio/f...), which gets sent over the same socket that the client established, client receives the request, does the file operations locally and sends the results back over the socket to the server, and server then sends them to ffmpeg.
ffmpeg has no idea its not interacting with a local file system
tonymet 1 hours ago [-]
very clever and thanks for explaining. for gpu-bound processes, which are common ffmpeg use cases, this is a great approach
deadlyllama 11 minutes ago [-]
I've been meaning to build exactly this for a while, and my waiting has been rewarded by someone else doing it!
Had you thought about using FUSE on the server side, rather than patching ffmpeg? Like a reverse sshfs? That avoids patching the ffmpeg binary, which allows usage of wierd and wonderful ffmpeg binaries with other people's patches.
I'd be interested in seeing how well it works with SBC GPUs - many have hardware decoding and encoding, and their vendors love to fork ffmpeg.
steelbrain 6 minutes ago [-]
I've explored sftp since ffmpeg has built-in support for it (-i sftp://...), but the support is quite buggy in code, I hope to submit some patches upstream to be able to change it. FTP in contrast seemed much more stable, at least looking at the code. FTP had some other shortcomings that made it undesirable for my usecase.
That was the one motivation, the other one was that it would require rewriting arguments going into the server. What you're describing was essentially what ffmpeg-over-ip v4 (and its earlier versions!) was, and the constant feedback I'd heard was that sharing filesystems is too much work, ssh servers on windows and macOS are a bad experience, people want to use a bundled solution.
Forking ffmpeg was no easy task! Took forever to figure out the build process, eventually caved in and started using jellyfin build scripts, but that has the downside of being a few versions behind of upstream HEAD.
ffmpeg-over-ip patches ffmpeg and only needs one port open for the server, then you just run the binary, no mounts needed at all.
throwaway81523 57 minutes ago [-]
This sounds like a nice project, but is it a Show HN without putting that in the title, because of the stigma that Show HN has recently acquired?
steelbrain 54 minutes ago [-]
You're going to have to chalk this up to human error, in the excitement to post it, I omitted Show HN:
naikrovek 2 hours ago [-]
This is software which basically replicates what Plan9 gives you out of the box.
Dammit I really wish Plan9 had taken off. It isn’t perfect but it does a much, much better job of helping me run my applications in ways that I want.
If anyone doesn’t already know, one method of Plan9 remote access is to “cpu” into a remote machine which has the hardware you need. Your local filesystems go with you, and your environment on the remote machine consists of your local filesystems mounted to the remote machine, but only for you, and all applications you run in this context execute on the cpu of the remote machine and have access to the remote machines hardware, but your local filesystems. Imagine SSHing into a remote machine and your entire environment goes with you, development tools and all. That’s what Plan9 does for you.
So if I “cpu” into a machine without ffmpeg, but with a GPU and I run ffmpeg, not only will it work, but I can tell ffmpeg to use a hardware encoder with a simple command line flag, and it’ll work.
toomuchtodo 2 hours ago [-]
Thanks for the update and continuing to share this project. What does the roadmap look like into the future?
steelbrain 2 hours ago [-]
I am thinking of adding a Windows application with an installer and a tray icon that you can use for some basic settings like changing port or password, or toggling automatic startup.
For linux, I am thinking of adding convenience helpers around systemd service installation
toomuchtodo 2 hours ago [-]
Very cool. Peertube supports remote runners [1] [2], might take a look for inspiration. As a distributed compute enthusiast, big fan of of this model for media processing.
Very cool! Thank you for sharing! I didn't know this existed, so now I'm curious how they solved it :)
My usecase is just-in-time media transcoding, I'll see if PeerTube remote runners support it
pastescreenshot 1 hours ago [-]
This is neat. The thing I would most want in the README is a benchmark section showing where it wins and where it does not. My guess is long GPU bound transcodes look great and tiny file churn workloads probably do not. Having that boundary spelled out would make adoption a lot easier.
FFmpeg is mountains of extremely complex C code whose entire job is processing untrusted inputs.
Choosing to make such code network-enabled if you can't trust your inputs, I would recommend to sandbox if at all possible. Otherwise you are asking for trouble.
The usecase for something like this is when you control both sides, server & client. There is some basic HMAC auth built into each request.
> I would recommend to sandbox if at all possible.
Since the server is a standard binary that doesn't need any special permissions, you could create the most locked down user in your server that only has access to a limit set of files and the GPUs and it'll work just fine. This is encouraged.
I'm excited to show case an update to a personal project of mine. Its called ffmpeg-over-ip and it allows you connect to remote ffmpeg servers, so if you have one machine with a GPU (could be your windows gaming laptop, gaming PC, a macbook?) and a machine (or VM, docker container etc) without a GPU, you could use the remote GPU to do GPU-accelerated video conversion.
The way it works is pretty neat, there are two components, a server and a client.
- The server (has the GPU) comes with a patched up ffmpeg and listens on a specified port - The client (without the GPU) connects to the server, takes the file IO requests from the server and runs them locally.
ffmpeg doesn't know that its not dealing with a local filesystem, so this approach works with multiple inputs or outputs like HLS, and is perfect for home media servers like Plex or Jellyfin or Emby.
One server can offer up its GPU to as many clients as needed, all clients offer up their own filesystems for their requests, the server also comes with a static build of ffmpeg bundled (built from jellyfin-ffmpeg scripts) so all you have to do is create a config file, set a password and you're good to go!
It's been about a year and half since this was last submitted (https://news.ycombinator.com/item?id=41743780). The feedback at the time was around the difficulty of sharing a filesystem between the machines so that should no longer be a problem.
This has been really useful in my local setup, I hope you find it useful. If you have any further questions, the website has some FAQs (linked in github repo), or you could post them here and I'll answer them for you!
Thanks!
Was really impressed by your work on Pundle. (It was an amazingly fast HMR dev environment - much like Vite today.) Felt like I was the only one using it, but it was hard to walk away from instant updates.
I built this macOS app that allows you to use any off the shelf wifi camera as your webcam with Zoom, Microsoft Teams, etc. It has lower latency than OBS, VLC etc based on my testing, its Swift-Native and pretty lightweight.
It was built mainly for my own team so they don't have to run long wires of USB cameras or pay a lot of money for a "wireless webcam". I hope you find it useful!
- https://github.com/steelbrain/ffmpeg-over-ip/blob/main/fio/f... - https://github.com/steelbrain/ffmpeg-over-ip/tree/main/patch...
I am not sure they'll be accepted for upstreaming, but in exploring the options, I noticed ffmpeg has sftp:// transport support and there were some bugs surrounding that. I do intend to publish some patches for those.
ffmpeg-server runs a patched version of ffmpeg locally, ffmpeg requests to read some chunks (ie give me video.mp4) through our patched filesystem (https://github.com/steelbrain/ffmpeg-over-ip/blob/main/fio/f...), which gets sent over the same socket that the client established, client receives the request, does the file operations locally and sends the results back over the socket to the server, and server then sends them to ffmpeg.
ffmpeg has no idea its not interacting with a local file system
Had you thought about using FUSE on the server side, rather than patching ffmpeg? Like a reverse sshfs? That avoids patching the ffmpeg binary, which allows usage of wierd and wonderful ffmpeg binaries with other people's patches.
I'd be interested in seeing how well it works with SBC GPUs - many have hardware decoding and encoding, and their vendors love to fork ffmpeg.
That was the one motivation, the other one was that it would require rewriting arguments going into the server. What you're describing was essentially what ffmpeg-over-ip v4 (and its earlier versions!) was, and the constant feedback I'd heard was that sharing filesystems is too much work, ssh servers on windows and macOS are a bad experience, people want to use a bundled solution.
Forking ffmpeg was no easy task! Took forever to figure out the build process, eventually caved in and started using jellyfin build scripts, but that has the downside of being a few versions behind of upstream HEAD.
https://github.com/joshuaboniface/rffmpeg
You can mix and match operating systems, macOS, Windows, Linux, you do not need sudo privileges.
rffmpeg needs a shared file system which could be a huge pain to setup: https://github.com/joshuaboniface/rffmpeg/blob/master/docs/S...
ffmpeg-over-ip patches ffmpeg and only needs one port open for the server, then you just run the binary, no mounts needed at all.
Dammit I really wish Plan9 had taken off. It isn’t perfect but it does a much, much better job of helping me run my applications in ways that I want.
If anyone doesn’t already know, one method of Plan9 remote access is to “cpu” into a remote machine which has the hardware you need. Your local filesystems go with you, and your environment on the remote machine consists of your local filesystems mounted to the remote machine, but only for you, and all applications you run in this context execute on the cpu of the remote machine and have access to the remote machines hardware, but your local filesystems. Imagine SSHing into a remote machine and your entire environment goes with you, development tools and all. That’s what Plan9 does for you.
So if I “cpu” into a machine without ffmpeg, but with a GPU and I run ffmpeg, not only will it work, but I can tell ffmpeg to use a hardware encoder with a simple command line flag, and it’ll work.
For linux, I am thinking of adding convenience helpers around systemd service installation
[1] https://docs.joinpeertube.org/maintain/tools#peertube-runner
[2] https://docs.joinpeertube.org/admin/remote-runners
My usecase is just-in-time media transcoding, I'll see if PeerTube remote runners support it