A friend of mine found a video player, but a unique one, which only runs custom encrypted videos. I have been interested in video encryption for a while, but this one is different!
It’s a dotNET player that takes an encrypted video file and plays it. It has other mechanisms to prevent screen recording, but from my experience, it’s just a flag and can be bypassed effortlessly, and even hardware acceleration can go through it.
Enough talking. Let’s reverse!
Detect it easy on the main executable.

As stated in the image, it’s a dotNET app, which is an interpreted language with a lot of info, so let’s decompile it using dnspy (or any other dotNET decompiler like dotpeek for example)
By looking through it, we find a function that handles video data
It parses a custom header to get the video info (length, name, etc.)

Then, it decrypts the video header (100 bytes), but the rest of the video is unencrypted.
It creates a hash of a constant string and XORs the header with it.

But wait, there’s more!
In an updated version, they edited the algorithm, so lets take a look

It now passed the file path to an unmanaged DLL
Loading it through IDA and tracing the close, we find the following

For simplicity purposes, the DLL has an MD5 hashing function I named MD5Hash and it gets the packageID as before and hash it and then concat random strings to it and then hash it again
And it does that until it gets to the final hash
That’s so boring, actually, so I’m using another method
Since both versions use XOR cipher that doesn’t stop me from doing XOR brute force of the known MP4 iso base media file format (isobmff) to get the key

If this article helped you, please share it with others!
Some information may be outdated