Greetings Today i would like to announce a new project that's been in the works for quite some time by myself and @Slacor . Project VideoDiff . An open source program written in Python for detecting visual anomalies including (but not limited to) temporal dithering using the OpenCV library. Unfortunately it's often not as easy as doing a passive screen recording and using that for input because most visual anomalies that cause issues are at the GPU output stage, but when paired with a lossless capture card such as the DVI2PCie , it renders VideoDiff into a useful investigation tool for investigating visual anomalies across many types of devices and operating systems. Here is a demonstration of VideoDiff being able to capture temporal dithering that was explicitly enabled on a Linux system with an AMD Radeon M370X graphics chip. You can see the difference between the blue, green and red color channels and the "mask" method which overlays the differences of the previous frame onto the current frame. Demonstration (should be relatively safe to view) Here is the source code. More information and tests to follow in the future. Welcome to a new era

Fascinating, I look forward to seeing where this goes.

This looks like really impressive work and a potential game changer. Roughly how much work went into getting this working?

Edward Roughly how much work went into getting this working? I first ran some image subtraction and comparing pixel value tests using a webcam as a "stand in" (the noise "simulates" dithering) around 2 years ago, but I got the PC with a lossless capture card and resumed working on this in January.

Nice work. Are you using OpenCV for its video decoding, or are you doing any machine learning stuff? I've been wondering if there is a machine learning solution to finding the original dithering algorithm that's been applied.

Seagull Are you using OpenCV for its video decoding, or are you doing any machine learning stuff? I've been wondering if there is a machine learning solution to finding the original dithering algorithm that's been applied. Just for video decoding and static computation based on the frame pixel values, but I'm all ears 🙂

Just to understand how this works, in a nut shell: it takes an already-existing video file, plays it back and makes changes visible by coloring them? So it could be used with any capture card and existing video files? The cheaper cards don't capture raw but use downsampling. Would temporal dithering still be visible? Would be great if there's a real cheap card anyone could afford.

KM Just to understand how this works, in a nut shell: it takes an already-existing video file, plays it back and makes changes visible by coloring them? It can also do a realtime input if the card cooperates, but yes KM The cheaper cards don't capture raw but use downsampling. Would temporal dithering still be visible? Would be great if there's a real cheap card anyone could afford. Unsure. I got the best card I could to make that less of an issue.

JTL Its only wondering, I have no idea if its feasible or how to do it. Even if its not as absolute as reverse engineering the algorithm, it might be useful to quantify the degree of randomness. As I have pontificated before, perhaps a more random dithering pattern is less likely to create aggravating visual patterns. Afterall, my personal experience has been GPU dithering generally bad, monitor dithering generally ok. It would also be helpful to be able to determine if a GPU is capable of multiple dithering algorithms, and when they are used. Idle thoughts aside, I would like to learn OpenCV as it might be useful for my new job. This could be an interesting place to start. Anyway, back on topic. Looking forward to seeing how your findings compare with mine. I did not use OpenCV to decode, instead I used VLC player to screenshot each frame, and then decoded the .png files produced using a rando library I found. Be interesting to see if that makes a difference. You are of course welcome to my capture samples, though I'm not sure how to share them. They total 20GB now, and the internet here isn't amazing (0.4MB/s upload only).

Seagull You are of course welcome to my capture samples, though I'm not sure how to share them. They total 20GB now, and the internet here isn't amazing (0.4MB/s upload only). If you have a computer that can be left on to host them I have some idea. Feel free to email me (jtl at teamclassified dot ca)

JTL Unfortunately I don't have a pc I can leave on. I might have a 1tb onedrive account that I can use to share. Will get back to you.

Project VideoDiff

Slacor

I have unlimited google drive. If it's uploaded to google drive and available, I could re-share it indefinitely.

JTL

Slacor I have my own servers that could be used at some point.

KM

Did you find anything unusual yet, for example with DirectWrite font rendering, Firefox Quantum, "Linux eye strain", or by plugging in a known-bad graphics card in BIOS mode?

Slacor

JTL Certainly. My thinking was some kind of write up on the wiki of the sample's conditions, testing different scenarios and seeing what happens, with result and source files.

Seagull

Put them on Google Drive as well, sent you an invite @JTL can invite you @Slacor if you want them too?

JTL

Slacor Me too. When I have the time I'll run more tests with my own equipment and testing setup as well.

Lauda89

Can you do this test also on windows or only Linux? Because I would understand if you can see something weird on W10 1909! I think that the problem is not just the OS itself but a combination with the GPU, because the 1809 is fine on my laptop (intel HD 5500) but not on my desktop (AMD 480X). The 1803 is fine on both setup.

Seagull

Lauda89

I should be able to capture something like this as I have an RX560 to hand, but I am pretty confident I won't be able to find anything insightful for you. Cards which dither, like yours, seem to do so regardless of the version of windows. Its possible its dithering differently to what it was before, but if I understand correctly that's capability that neither mine or JTLs software can yet perform.

JTL

Seagull but if I understand correctly that's capability that neither mine or JTLs software can yet perform.

Correct

Seanicus

Great work guys. Keen to follow this.

degen

When I tested WIndows 7 vs 10 on my e-ink display with HDMI (Dasung Paperlike Pro), dithering patterns appeared different.

JTL

degen What hardware of the system under test?

degen

JTL Sorry for not getting back sooner. Redoing some testing. Maybe it can give some ideas for using videodiff. Not feeling so well but I will get to it!

AGI

JTL Hey, I am not sure if my question makes sense...could your method be used also to study the effect of lowering display resolution on dithering?

JTL

AGI Potentially.

martin

Amazing, great work. Is there any chance it will be more easily usable as a simple exe or an app for mac or iphone even? Im not that good with using source code.

JTL

martin Amazing, great work. Is there any chance it will be more easily usable as a simple exe or an app for mac or iphone even? Im not that good with using source code.

Outside of attempting to visualize an existing video it requires special hardware.

EXE might be possible

JTL

degen Okay, good luck

JTL

Donux Funny, but lets keep things on topic

Alyosha2001

I found this listing on ebay, seems to be more available. Could you make it available as exe, as I don't know how to use source code?

https://www.ebay.com/itm/325232137678?hash=item4bb95895ce:g:-R4AAOSw0f9iqzW0&amdata=enc%3AAQAIAAAA4A4gAgBNzlqER3oRXdj55ClSCtcmTbj%2B1mXlHsVYcgNXv7K2gJu%2FVPxrTW8dV4FNAheZI709WLKr4TuVUhGx0QBX8BCM5my48sBugTC5I3we0VzFSjKmP3Y0IQPJvQ8C7f9tYv25JVDNL0fy3Mw8eV%2BVSOBYPnbJBw85Sfc8WTIfnHOmlLZvsI0zEKSpgbJ7b4uYfsFLvs8ANzGFr6sUa5FwNJm%2FQ3rlKGx2TDPDXHdbkF3PNIA6lkJ7jZou0hTdxD%2BJXHTkwhr1svDwl2Qh0x5qdy80Lfuw8zB0SilUksLv%7Ctkp%3ABk9SR6yeh9H_YQ#readMoreDesc

JTL

Alyosha2001 I found this listing on ebay, seems to be more available

I've seen that listing before. One thing to be aware of is that does not include a case bracket so you need to be mindful of the weight of anything connected to the card by the DVI port, also the weight being pressed against the card in the PCIe slot. I have another unit of the same model with bracket so it might be possible to fabricate a bracket of the appropriate size with some effort (metal work).

Could you make it available as exe, as I don't know how to use source code?

Releasing VideoDiff as an EXE might be possible, however something to be aware of is that VideoDiff is a command line application undergoing rapid development as a research project, so it's not some friendly "point and click" GUI. As a result you might want to familiarize yourself with its internals anyway. That being said I'm available for assistance and it's not that hard to understand when you get the hang of it.

« Previous Page Next Page »