I have been through all this with fitting an aftermarket dab radio to my 2007 van, I done what you did by swapping the yellow and red live wires, and believe me it will just flatten your starter battery. Ours is a basic pioneer but I would think most will be the same when you turn off the ignition the head unit is still live and yeah the head unit will have a on/off switch but the unit will still be on. I went through most of last summer with this happening to my van because I couldn't be bothered to sort it until the day we were heading off for a two week camp, van hadn't been used for a good while. Battery was that dead I couldn't even open the sliding door

took a full 24hrs to charge the battery with a ctek battery condioner, I didn't want to use a fast charger in case it fried the battery thankfully the battery is fine and I think it was all down to the ctek.I knew the radio was still live because I have a adapter that connects to an adaptor socket and gives you the charge of the battery wether it be starter or leisure battery, three colour led display green is good, yellow starting to lose charge but generally fine and red the battery needs help pretty soon
Only thing I would recommend is to start again and use the fuse in position 19 on the front dash fuse board, (connect a piggy back fuse, simplest way) connect a new wire to this and wire this to your switched live on your head unit. You will have to switch the yellow/red wires round again on your quadlock.
Fuse 19 is a switched live a thin brown/green wire.
Hope this helps and don't worry it's a really easy job.