Deunan

Spot differences between two pictures:



The first one was taken on system with Catalyst 13.1 WHQL and the second one on 13.2 beta. Day wasted on fixing this particular "bug". And before someone points out "This is what you get for using beta drivers" - there is a reason I need to be on 13.2 or newer.

So, there is this guy who keeps asking about DX11 renderer progress and I figured I might as well explain it with some pictures :)

Direct3D 11 is very different to D3D9, and this is the source of my problems. While I want the functionality it offers, I need to properly rewrite every part of the code. In fact the change is so big I created a separate project that is based on NAOMI emulator code but all it does is render one frame of previously captured data. Makes my life so much easier.

D3D11 is a functional extension to version 10 interface except you work on context rather than the device itself, which is only used for creating new resources. This lets you create command lists for things like deferred rendering, though I stil use immediate mode. Unlike in 10, the reference counters are incremented by various Get* methods and this is more like it was in D3D9, so the behaviour was clearly reverted. And for the better I suppose - makes much more sense this way.

The main problem is the grouping of various settings that I need to change individually, this was much easier on D3D9. While the new approach certainly speeds up things for games, I cannot make any assumptions about the 3D data I'm processing and I can't sort it nicely either, so I have to prepare many object with all possible settings well in advance and select the right one when the time comes. Doesn't seem that complicated, eh? Well, it's actually is much more bothersome than it sounds.

Anyway, I decided to start with the basics and then, once it all more or less works, try to add the layer peeling system for per-pixel depth sorting. There will be another major obstacle on the way, namely the fact that I can have different blending operations mixed in together. So I will need to store the blending instruction data along with the pixels and have pixel shader combine the fragments properly. Which is not going to be easy or super fast :) Still, no point in worrying about that now. It's either that or full software rendering if you want proper PVR2 emulation.

Here are some sample screenshots I took along the way, so that you could see there is progress. Note that I already ported and tested the text display class - though with just one frame being rendered the log is much better target for debug output.









These two are different becase I had to change the capture data format a few times:





Anyone wants to guess what game is this? Should be pretty easy by now :)


EDIT: We have a winner! It's Zero Gunner 2.

Name that cart:



That's right, -MAZAN- Flash of the Blade :) Many thanks to Hans from Sweden for lending it, and to Tomas for doing the actual dump. Unfortunately I broke JVS layer in my code so I'm unable, at the moment, to run it in emulator. It boots, sure, but dies with "I/O PCB ERROR" shortly after. So, the best I can show right now is this (don't quote me on the program checksum though, I got a bit creative with it before I made this screenshot):




Wait a minute, I do have a NAOMI2 box, right? A few emails later and some photos & manual exchanged, I've added another profile to my JVS I/O and it now spoofs the custom boards that came with the game. Still no go as there is a sanity check run on the sensors and I couldn's pass that, but at least I got the game test mode working:



Another hour or two spent patching the game program and finally this:



Yup, looks ugly. Sorry, the photo wasn't that great in the first place and you can tell some stuff is missing - like, the sky in background and half of the textures. I either messed up de-protecting the game or it just doesn't like being run from DIMM, much like HOTD2. Well, DIMM-enabled HOTD2 runs perfectly in emulator so I suspect addressing problem that only manifests itself on hardware. Whatever the cause, it's fixable - as soon as I figure it out :)


I've been also asked to do something about GDI format. The main issue is it keeps all the tracks in separate files and there are games with tons of digital audio. It doesn't carry any information about the game title either (especially if it's not an English one) or ring code - sometimes the only thing that allows you to tell GD-ROMs apart. Well, unless you can read the disc contents just by looking at it :)
It's primary purpose is to store Dreamcast-playable discs so it can handle GD-ROMs and MIL CD-ROMs (mostly because CDI format used so far is not exactly free, legally speaking). It's not all-purpose substitute for all optical media there is, thus simple and fast where it counts.

This is probably going to raise a few eyebrows in "What, another format?!" fashion, but don't worry. I'm not dropping support for GDI/CDI and ISO files, just adding one more. Whether or not this new format will become even remotely popular is not really my problem, the people who want to use it will do so. Some folks will also ask "Why not CHD then?" - well, I think CHD is great for MAME. And let's leave it at that.

Here's how a converter looks like (still WIP). Notice that you can store a small image within the file (256x256x32) so that it would be easier to identify the game. By default the image is that of GD texture used by BIOS in CD-player mode. Some games don't have it though, or there might be a better suited cover scan or photo, so it's possible to load an external one. It's completly optional but recommended.

I'm pretty busy lately so I don't have time to answer any questions in the comments, sorry. I'll just address most pressing matters in this small update.

Yeah, I'm aware that MAME has got some more NAOMI games and that Makaron doesn't recognize the ZIPs. Well duh, I need to know the filenames first and I use MAME sources for that. Unfortunately you can't just unpack those ZIPs and create continuous images - or rather, you can, but since public versions of Makaron don't contain the necessary decryption keys the protection will not be emulated. Hence no joy. Support for these titles will be added soon though.

Apparently there's yet another GUI for Makaron :) Saves me some trouble for now I guess. One of the ideas I have is to just drop the internal one completly as creating a nice GUI in pure Win32 is such a PITA. I could make an external one in Delphi or something. Hotkey-based disc swapping for Dreamcast seems to work for most people.

Work on DX11 stuff is progressing but truth be told it's a bit like switching from DirectDraw to Direct3D - nothing works anymore and needs to be rewritten plus tons of new stuff to take in. DX9 was pretty matured so documentation wasn't a problem - can't say that about DX11 I'm afraid, there are some demos/sample code but that's not enough in many cases. Fortunately Geometry Shader is simple so I can focus on Compute/Pixel Shader and OIT. Still no idea how costly it'd be to switch shader programs, I keep doing that since I have no choice but maybe a longer PS code with dynamic conditionals would actually be faster? We are talking 2009+ hardware after all.

I've also been messing around with x64 code emitter though it's just for fun now. I still need a 64-bit capable compiler first :) While it's possible to integrate that into MSVC Express version it's not exactly straightforward - and I'm not paying for STD/PRO out of my pocket just for that.

Also having some fun with soldering iron, but nothing you people would care about I suppose.

It seems that CheckInterfaceSupport is broken for DirectX 11. This is what I'm doing:

HRESULT hr = pAdapter1->CheckInterfaceSupport (__uuidof (ID3D11Device), NULL);

The result is DXGI_ERROR_UNSUPPORTED for Radeon 5770... Creating ID3D11Device with D3D_FEATURE_LEVEL_11_0 obviously works, as the card does support it. There is no such problem when asking for ID3D10Device.
The plan was to enumerate all adapters present in the system and reject those that do not support Direct3D 11. This bug effectively prevents it from working. I don't know, maybe it's driver related, but very annoying nevertheless.

So yeah, I switched to 64-bit Windows 7 Pro and I also bought a new graphics card. Rather than spending money on a new i3/i5 CPU I overclocked my E6600 to 3.0GHz, that should last me another year I hope. With this change I will be slowly moving away from Direct3D 9c - as I explained many times before it has too many limitations to properly emulate PowerVR2 chip. Don't worry though, I'm not going to drop D3D9 support immediately but it will be removed someday. Makaron doesn't need all that much GPU processing power, rather the new features, so it's quite possible that the new CPU+GPU hybrids from AMD will be fast enough to run it full speed. And those are supposed to be pretty cheap.

I changed a few things in T12 core and I'm finally quite happy with how it works so I'll probably release another version soon. That would be the last of Test 12 series I suppose. The only thing preventing me from doing so is one last issue with the multithreaded renderer, it causes random crashes (though some systems seem more affected than others). It's a pretty complicated issue though and cannot be fully fixed without major changes that would take considerable more time. For now I just added an option in the INI file to disable multithreading (not completly, just this part) and while it slows the emulator down it's not as bad as the crashes.
Anyway, stay tuned. I want to make a release before I mess up the code again with my experiments, and I have quite a few ideas to try out :)


UPDATE: Here it is, Test 12/5. There's a small ReadMe included so be sure to read it. No Atomiswave yet since I didn't think T12 core could be salvaged, so I didn't bother with it. I wasn't exactly wrong there, with all the changes I made this version could easily be called Test 13 :)

Anyway, other things of note (from the top of my head):
* Current renderer needs to go. I have a replacement of sorts prepared, as a part of software renderer project. I'm pretty sure it will be a bit slower but that remains to be seen. I need the new framework anyway to support both D3D9 and D3D11 at the same time.
* SH4 recompiler could use some sort of abstraction layer to handle the optimizations better. Again, it'd probably end up being marginally slower but more manageable and could in future be paired with x64 emitter.
* I'm going to drop support for non-SSE2 CPUs soon, mostly because it'd be easier to code the new recompiler. Quite frankly any CPU without SSE2 is not going to be fast enough to handle Makaron anyway.
* Windows XP and Direct3D 9 support is going to stay for now but once D3D11 renderer starts working properly I will focus mostly on that. While I realize that droping DX9 will upset XP users, I'd like to point out that Makaron always required a modern PC. I havent introduced any major changes lately but before that Makaron was the first Dreamcast emulator to require shaders, then it would work only with Shader Model 2 or higher. In return you got faster palletized texture support, shadows and fogging. Now it's going to be DirectX 11 :)


EDIT: Seems like 640x240 interlaced modes are not upscaled properly. One way around it is to change cable type to VGA. If the game doesn't support VGA cable you're out of luck, use mode 4 or just go back to 640x480. This should never happen on NAOMI since by default it runs in 31k setting. I fixed that already but I'm not pulling the files unless there is a bug that can't be worked around.


UPDATE 2: That bug I wrote about? Now you're seeing it's effects first-hand :) Oh well, guess I better fix that one since I didn't add an option to skip DX11 init.
Redownload T12/5 from here, this time when Direct3D 11 is present on the system but actual device can't be created it will just silently ignore the problem. This was the intention after all. And I also added workaround for the scaler issue.

In a valiant (though doomed to fail, really) effort to clean up the mess that I call my living space I've made some shelf space by throwing away old junk. I'm the "collector" type, unfortunately, which means I tend to keep every bit of electronic or mechanic trash in hope it can be later salvaged for parts and whatnot. Every now and then I read about old people going through trash cans and bringing home anything they find interesting and can carry - and hope this is not how I'm going to end up...

Anyway, I got rid of some old and pretty much used up electron tubes. The radio those were meant for is long gone, and if I'm ever going to put something together using that technology I'll stick to noval/loctal tubes. Yup, still got some of these. There is just something magic about DIY dual-triode AM radios :) Though I guess AM is going to be dropped someday, just as morse code got...
The point here is that shelf space I mentioned. I decided to fill it up with something, obviously, and one of the things that ended up there is brand new Playstation 3. This is "new" in more ways than one, my first console that was not preowned and half the price when I got it. As you can guess it's been eating away some (OK, maybe most) of my free time now. And before you start the usual X360 vs PS3 war, I only went with SONY for the few exclusive games that I wanted to play. A difficult choice seeing how I most likely won't get to play Ace Combat 6. Or Tales of Vesperia. Hell, I'm even hesitating with Bayonetta now.

In fact I think X360 is a bit more powerful hardware - sure it can't compete with CELL SPUs in terms of raw SIMD/streamed data crunching performance but then again the single PPU unit is the very weakness of the PS3. That and the split memory, especially when you're planning a multi-platform launch. And it turns out the GPU is somewhat lacking as well, requiring SPUs for vertex processing. Not that it matters much for the end-user, although many people seem to insist on comparing the guts rather than games. Or should I say, the quality of games and I don't mean graphics here - I'm not really a fan of console FPP shooters (a.k.a interactive movies) even if I always liked all Metal Gear Solid. The first Modern Warfare wasn't bad too, especially Death From Above :) Other than that I'm sick of heavily scripted gameplay sequences, so no Uncharted for me either. I kinda like the idea though, I'm just bad at the game having used mouse and keyboard for FPP games all my life.
To prove my poing, I only played MGS4 for like 15 minutes and switched to other games. Fat Princess, Resonance of Fate and Valkyria Chronicles mostly, with some Eternal Sonata, GT5 Prologue (I suck at getting my RX8 through tight cornes using the pad) and few others. I would play those games on PS2 as well, even with inferior graphics. I really fail to see what is so great about these "next-gen" consoles, especially that both X360 and PS3 struggle at 1080p output.

In the meantime I've messed around with Dreamcast side of Makaron, had a few good ideas too. Sadly the new WinCE recompiler is not ready yet, I underestimated the amount of work needed here. Though on the bright side it seems the changes I've made so far don't seem to hurt performance.
The fundamental problem T12 series had was it's inability to properly regulate emulator speed to match given PC hardware. Problem was it always worked more or less OK for me and so when my Japanese friends pointed this out I didn't quite understand what they meant. And then I changed some code bits, a few parameters and got such a massive speed drop (varying from run to run) that I couldn't belive at first. Turns out the emulator can actually make things worse for itself while trying to optimize the main loop CPU usage... I changed that but it's still somewhat broken. I have to run some more tests but I suspect DirectX plays foul. It's partly my own fault, I should not be calling things like GetCurrentPosition so often in the main loop but there aren't that many other options. You can't properly micro-manage threads in Windows, you either go for a loop that does mostly nothig (hence uses 100% of a CPU core) but has very little latency, or you let it sleep when idle but the shortest time you can specify is 10ms. And that isn't even guaranteed, there is nothing stopping the OS from making it 100ms from time to time. Or worse.

I'll probalby just add another ring buffer for mixed samples and decouple the whole thing from DirectSound completly, it's not that big of a change. The downside would be the AICA will now use only HPET so there is a chance this will eventually desync with actual sampling rate so badly there will buffer under/overrun. This might result in brief audio distortions but I don't expect those to be noticable. We'll see I guess.
This just goes to show that coding for a PC can be difficutlt, especially when you try to make a realtime system. With so many different hardware configurations and OS issues it becomes even greater challenge than making the emulator core itself.

Oh and by the way, I found some interesting blogs recently, mostly hardware related - down to the silicon level. Some great photos of bare chip cores with explanation of what is what. Here, a couple of links:

http://www.flylogic.net/blog/
http://www.bunniestudios.com/

Stuff like that reminds me that I'm just a small fry and there are guys (and girls?) out there doing serious work. I'm too lazy to solder ca. 60 protection diodes for my FPGA kit to able to peek into M1 encryption. But I'm getting to it :) As for the chips themselves, without a a good microscope all I can do is this:



The noise is mostly due to dithering from color range reduction, and I could get a better magnification but it's a borrowed hand-me-down microscope for biological samples, so it only provides light from below. And don't ask how I got he shot with my compact camera :P

Oh, and there has been a lot of spam in the comments lately, most of it from LJ spambot accounts. If this gets worse I will have to switch to a nastier censorship mode, that will include logged in users. Let's hope it doesn't come to that. Ah, I miss the days when I had the comments fully open to anonymous posters...

EDIT: And two more, slightly bigger pictures taken today. You can actually see the layers here now, though keep in mind it's '74 tech, those are but visible to the naked eye :)

To answer questions that have been popping up recently: No, I have not abandoned Dreamcast. It is true that lately I was focused mostly on NAOMI but that was due to all the work done on cart based games. It would be a waste not to implement these changes right away.

It's actually harder to get Dreamcast working right. NAOMI has fewer games, none of which uses WinCE kernel, and except some JVS woes the emulator base is pretty much complete now. What's left are NAOMI 2 extensions and maybe the comm board emulation. Renderer issues aside of course.
Let's be frank here, Makaron T12 had some major changes in it and didn't turn out all that great. What's more, there are still two major issues left unresolved:

1) Disc swapping
2) Proper fullscreen support

These two are very closely related. See, Makaron does support disk changing while running but the GUI won't allow you to do anything of the sort. And if you were playing fullscreen, you'd have to switch back to windowed mode to even access said GUI in the first place.
After I promised I'd bring back the ALT+key game image switching I realized that hacking T12 is pretty much pointles. I should focus on making it work as it's meant to. Sorry Yuki :) So, Dreamcast Test 12 branch is dead, I'm doing a major rewrite for T13.

I figured I only need to add GUI for the most important options, the rest should not be touched anyway unless you know what you're doing. Then there's the fullscreen and aspect ratio issue. I still haven't decided if I want to use slower but easier fake mode, or the real deal. I could perhaps do both but that means more code and obviously I don't like it :) Right now Makaron is unable to recover from a "device lost" situation and while it could be worked around, it'd be very ugly.
Another thing is WinCE support. There is a different way of doing MMU address translation, something I discussed long time ago with Nathan Keynes (lxdream author). It's much faster for Linux/FreeBSD but it remains to be seen what speedup, if any, I could get with games. It's a big change but fortunately Makaron is already half way there as is. This is something I actually look forward to, not a mindless bug hunting.

This kinda brings me to another topic, that is nullDC source code being opened. I'd hesitate with word "freed" here, it's not exactly as simple as some people think. Looking at the issues page it's obvious that there are still only two real maintainers: drkIIRaziel and PsyMan. I don't think this was the intended result. Whether or not someone picks it up or the original authors loose interest completly remains to be seen - but it's what I always say, not that many people are interested in writing an emulator. Source code being closed is no excuse, after all I started from zero on my own so anybody can do that. On the other hand, once the code is open you don't really see an army of helpers, now do you... Hacking in a feature or two does not count as proper development.

Since I touched the subject of features, I'm still against keyboard based gamepad support so don't get your hopes up. Get a wired X360 pad, it's the closest thing to a DC controller that can be hassle-free connected to a PC. I'm against Microsoft hegemony but having some standards put in place really makes my life easier. The hoops I need to jump through to get around bugs in DirectInput device drivers... ugh.
I'm also not going to add any additional texture filters, you don't like big pixels, go play a recent game and don't bitch about '98 graphics being poor. I might add full-screen filter effects though, depending on how bad it looks on widescreen LCD :)

So... when is Test 13 going to be ready? No idea. I might be forced to drop some of the changes I want to make in favour of faster release (like, say, integrating input plugins back to main executable). The software renderer is not going to make it in and thats pretty much certain, I only started playing with it. The rest... I guess we'll see about that.

Incidentally there are no waves to speak of. And no drive board. But there are dolphins.





EDIT: Removed that bit about JVS. It's not part of the emulator and was only causing confusion. Sorry for poor wording.


UPDATE: New DIMM tools set with v6 patcher. I hesitated but in the end the "leaked" Asian Dynamite image is also supported. It's not a clean dump but I guess you guys couldn't care less as long as it works. As a bonus it will also remove internal region check in GGXX games.
As for cart netbooting... Well yes, it's possible and no, not going to release any patchers. Every cart game needs it's own and while I could compile them together to make almost universal one, I'd need to distribute the thing with either trojaned out protection data or Naive's code - and that is something I simply won't do.
That said, I make exceptions for people who can prove me they own the carts. Mail me and maybe we can work something out.

You know, seeing how it's become something of a tradition on this blog, I'm surprised so many people fell for it again. Consider yourselves April Fools :)
Though I have to say, I rather liked the idea - it's hard to come up with something both silly and still somewhat believable.

Anyway, there isn't a NAOMI emulator for Nintendo DS. Not now, not ever. Let's look at the numbers here:
NAOMI has 2MB BIOS ROM, 32MB of main RAM, 16MB of VRAM for PowerVR2 and 8MB RAM for AICA. That's not counting various smaller regions like OCRAM, NVRAM, JVS and cart buffers, etc. Compare this to NDS lite that has 4MB of main RAM... It't not enough to even get my DSF player running :) Maybe the Dreamcast-only version would, since on DC there is only 2MB of AICA RAM, but then again you need space for the code and data as well. Speaking of which, the emulator itself requires large amounts of memory for recompiler code buffers and dirty memory maps...

I'm not very big fan of ARM architecture. It almost feels like CISC with all those control bits that change how each instruction is executed. And you loose some of that functionality and registers when switching to Thumb mode. I like SH4 better, it's only real weakness is immediate operands that are bigger than 8 bits. Those must be loaded into registers with PC-relative addressing, thus wasting precious cycles. Not surprisingly though it's not that much of a problem in typical code.
SH4 is superscalar, pipelined CPU that can execute two instructions at a time, including floating-point ones. It can do 4-element dot product in one and matrix multiplication in 4 cycles. Fused multiply-add is also available. In Dreamcast and NAOMI it's clocked at 200MHz and topped with some on-chip peripherials like timers, DMA controller and MMU. On the other hand, all NDS can offer is one ARM9 clocked at 66MHz and one ARM7 at 33MHz. Both have bus width to main RAM reduced to 16 bits, which pretty much means Thumb mode only. That's laughable by today standards and it would be even worse if not for the WRAM and cache memory.

There is little point in comparing PowerVR2 and DS 3D engine :) Many newer handhelds, while much faster on paper, can barely match the performance of good old PVR2. This will probably be over soon with the new generation of SGX chips and/or Tegra ones. The problem here is mainly the power consumption that has to be kept down and also design limitations such as reduced bus widths and smaller memories.

And let's not forget it's emulation we are talking about. It's all about converting data and code on the fly - a process that is always pretty expensive computation-wise. Then there's the basic rule of trading CPU speed for memory usage and the other way around. Most emulators work at acceptable speeds only thanks to extensive optimizations, some of them being pretty clever :)



Now, I stated on my bio page that every now and then I will blog about things not directly related to Makaron. With that in mind, and considering that I don't have a Playstation 3 and have never used it - with or without Linux installed - I feel like I'm entitled to bash SONY a bit. Yup, it's about the 3.21 update from yesterday.

First let me state that I fully understand SONY position on this. If I were their boss I'd order pretty much the same move. They made a nice, secure system and want to keep things that way, can't really blame them.
On the other hand removing an advertised feature (it only does gaming now :) and only because of some very early hacking attempts? Seems like a bit of an overkill, their PR will suffer. Slim users might not care for now but the question is, what will come next? I mean it's pretty funny to see an update that only blocks or removes otherwise useful functionality, and it's not really optional either considering you loose PSN access otherwise and possbily the ability to play future games/movies. This is a bit like hearing "yer money or yer life" while being held at gunpoint. While there are two choices here it's only theoretically.

There are ways to "fully crack" PS3. The most obvious is to obtain the master decryption key(s) stored safely inside CELL. Or there might be a nasty bug in one of the encrypted SPU programs that would leak some important data, or bettr yet, serve as decryption box. There might be a way to load games directly into RAM, bypassing the media checks - this one is tricky because with the code still encrypted we can't be sure what the checks are and when they will happen.

So, why remove Linux? I think it's because of RSX/SPUs lockout. It's there to prevent people from writing homebrew games and programs that might look and feel as good as SONY licensed products. Why? Because this is how SONY makes money. Games made with free (or at least cheaper) 3rd party tools would also be cheaper. You might think "but not as good" - well, possibly, but think about TETRIS. It doesn't take super-realistic 3D to make a good game - I for one find many of current generation games simply boring. It's less and less a game now and more an interactive movie these days... And don't get me started on PC DRM methods :)
This would of course violate EULA but that hasn't stopped people from trying before... If there is money to be made, things like that suddenly don't matter much.

I think the CELL encryption will remain secure for some time to come. Eventually some people will decide it's time to profit from pirated BD media and will start a manufacturing plant somewhere in Asia. That, or decap CELL to make PS3 modchips. Whatever. For now though SONY is afraid of the lockout being removed or bypassed. It remains to be seen if this move will help them, or actually motivate various hackers to try harder...