ps3xport Released by KaKaRoTo
On December 23, 2014, in Development, PS3, by kakaroto
Hello everyone!
Download:
https://github.com/kakaroto/ps3xport
Source:
https://twitter.com/kakarotoks
More needed info about Data Backup can bé find below
http://www.psdevwiki.com/ps3/Archive.dat
Read previous article below.
http://www.xxzer0modzxx.net/forum/index.php?threads/early-days-ps3-4-0-hen-by-kakaroto.46/
On December 23, 2014, in Development, PS3, by kakaroto
Hello everyone!
It’s been quite a long time and I’m very happy about that 
Let’s do the boring part first! This is my final release for the scene, I am not “coming back” or anything like that, so don’t get your hopes up, but I needed to release this so I’d be officially done. I have never actually announced that I’m leaving the scene but everyone figured it out. It wasn’t originally done intentionally actually, but life caught up with me, work, family, lack of time, etc.. so I had little time to work on the ps3. Also, my motivation was mostly gone due to not finding anything interesting anymore, a lot of drama and I’m not a huge fan of all the attention this all brings. I got into the scene because I was curious and I wanted to learn, and I have to say I’ve learned a lot of things these past years and it was an incredible journey, but as I had lack of time and started breathing, I realized that I’ve had enough of it so I left and I am very happy with that decision because you have absolutely no idea how much of a time drain and headache this was
Anyways, there was one thing I did just before I left, but I never got to release it, but today is your lucky day as it’s release O’clock where I am!. This release is a way to say Merry Christmas, Happy Holidays, etc.. to everyone, and a way for me to also say “I’m done for good, I don’t have anything left for you in a drawer somewhere”. I’ve wanted to release this for a while now, and I even made a poll on ps3hax back in March 2012 asking people if I should (looks like ps3hax is down right now so here’s the google cache version) and the general response was not to release it until it can be useful (when an npdrm workaround is found) with some people saying to release if nothing new happens in the scene.. and I think I’ve waited long enough now to know nothing new on that front will happen.
So.. since I’ve announced the release, I’ve seen a lot of speculation about what it is and what it could be.. a lot of people seem to think (or mostly, want/hope for) a downgrade method, unfortunately that’s not the case. I’ve seen some ridiculous suggestions too, like someone asking if it’s a way to run PS4 and Xbox One games on PS3.. I’m sorry to say, that’s not it either As I’ve said in a tweet shortly after, this is nothing groundbreaking, this is code that hasn’t been touched in 3 years, so it’s already 3 years old, but I think it’s still something that can be very useful to the community.
So here it is, I’m introducing to you : PS3xport! I’ve uploaded it to my github account here :https://github.com/kakaroto/ps3xport
What does it do? Well, it’s basically a tool for manipulating the PS3 backup data. When I say “PS3 backup”, I’m not talking about a “backup” of a game, no.. I’m talking about the full PS3 hard drive backup that you can do by going to “System Settings->Backup Utility” on your XMB. That creates an encrypted directory on your FAT32 hard drive which allows you to format your PS3 and then Restore it just like it was before. I’ve reverse engineered the file format and encryption and PS3xport allows you to create new backup data from scratch, or dump existing ones, or delete specific files from a backup or do a whole lot of other things to your backup folders. This gives you total control over your /dev_hdd0 and /dev_flash2 filesystems, which will let you install homebrew on any console, even if it’s the latest OFW version. Unfortunately, just like it was 3 years ago, you wouldn’t be able to run those homebrew apps you install due to the NPDRM ECDSA signature missing. If you have your IDPS though for example, it could let you restore a backup from one PS3 to another PS3 without losing any of your data in the transfer.
So.. what’s this about “your IDPS”? yes, the backup has two sets of files, some can be decrypted right away and some can’t because they are encrypted with your IDPS (your unique ps3 device id) which is why they can’t be restored on a different ps3. If you have a CFW, you can easily get your IDPS (I’ve written a small tool to do that, released on github, but apparently MM and Webman will also give you that information) and that will give you total control over your backup data as you would be able to decrypt and reencrypt it. If you have OFW and can’t get your IDPS, then you will not be able to dump/decode all the files from your backup, but you will still be able to create a backup that can be restored on your PS3 with no limitations (this means for example that you can restore a backup from a CFW into an OFW without any issues). I was told however that someone can get IDPS from OFW consoles and in light of this release, they might release their method soon, I can’t say more than that though, but be patient and good things come to those who wait
So my release is in two parts. First, the documentation of the file format was added to theps3devwiki so any developer can understand how the backup archive files are created and can create their own tools. Reverse engineering that format took months of work and I won’t go into too much details about what had to be done to figure out the format but it was an incredibly long and difficult task to do that I had a lot of fun in doing. The second part of the release is of course the release of the ps3xport tool. The tool is quite powerful and you can do a lot of things with it, but it’s a command line only tool and I honestly just tested it on Linux, it’s not really my job at this point to make a windows build, or make a GUI around it, etc.. but I’m sure it won’t be long before others in the scene pick it up and make a nice GUI for it and release windows binaries. I’ve written a nice README file so everyone can understand how the tool works and what it can do. I remember though that 3 years ago just before I stopped working on it, I wanted to add a “AddPKG” command to it which would just ‘install’ a pkg into the backup data automatically, unfortunately, I never got to do it, but it should be easy to do. While I’m at it, I’m also releasing a pkg extraction tool which I found in an old directory (cool thing is the -p option in it, try it…) as well which is a PKG extraction tool that uses the PagedFile mechanism (see below) to allow for very fast pkg file access with very little memory usage even for huge pkg files, any dev can probably mix those two together to add the AddPKG feature to ps3xport.
On the software front, ps3xport.c will parse the commands then use the archive_* API which is in archive.c. That will contain all the functions needed to manipulate the archive files. It uses a ChainedList which is my rudimentary implementation of a GList-like ordered list and the archive API also uses PagedFile objects which are pretty cool. PagedFiles are a wrapper around a file which allows you to read/write to a file using pages (I set it to 64KB per page I think) so it limits the hard drive access. The cool thing about it is that it has encryption and hashing built in, so you can just set the encryption key or ask for the file to be hashed, and whenever you read/write, the encryption will be done transparently, and the coolest thing about it is that you can actually seek in the encrypted file and it will still work (it recalculates the required IV whenever you seek). The encryption there works on the stream, so you don’t need to write blocks of 16 bytes every time (thanks to the paging of the data) and it has a cool ‘splice’ method which allows you to copy data from one PagedFile to another easily, so you could in theory re-encrypt a file using a different key using 5 function calls (open *2, set_key*2, splice).
Let’s do the boring part first! This is my final release for the scene, I am not “coming back” or anything like that, so don’t get your hopes up, but I needed to release this so I’d be officially done. I have never actually announced that I’m leaving the scene but everyone figured it out. It wasn’t originally done intentionally actually, but life caught up with me, work, family, lack of time, etc.. so I had little time to work on the ps3. Also, my motivation was mostly gone due to not finding anything interesting anymore, a lot of drama and I’m not a huge fan of all the attention this all brings. I got into the scene because I was curious and I wanted to learn, and I have to say I’ve learned a lot of things these past years and it was an incredible journey, but as I had lack of time and started breathing, I realized that I’ve had enough of it so I left and I am very happy with that decision because you have absolutely no idea how much of a time drain and headache this was
Anyways, there was one thing I did just before I left, but I never got to release it, but today is your lucky day as it’s release O’clock where I am!. This release is a way to say Merry Christmas, Happy Holidays, etc.. to everyone, and a way for me to also say “I’m done for good, I don’t have anything left for you in a drawer somewhere”
. I’ve wanted to release this for a while now, and I even made a poll on ps3hax back in March 2012 asking people if I should (looks like ps3hax is down right now so here’s the google cache version) and the general response was not to release it until it can be useful (when an npdrm workaround is found) with some people saying to release if nothing new happens in the scene.. and I think I’ve waited long enough now to know nothing new on that front will happen.So.. since I’ve announced the release, I’ve seen a lot of speculation about what it is and what it could be.. a lot of people seem to think (or mostly, want/hope for) a downgrade method, unfortunately that’s not the case. I’ve seen some ridiculous suggestions too, like someone asking if it’s a way to run PS4 and Xbox One games on PS3.. I’m sorry to say, that’s not it either
As I’ve said in a tweet shortly after, this is nothing groundbreaking, this is code that hasn’t been touched in 3 years, so it’s already 3 years old, but I think it’s still something that can be very useful to the community.So here it is, I’m introducing to you : PS3xport! I’ve uploaded it to my github account here :https://github.com/kakaroto/ps3xport
What does it do? Well, it’s basically a tool for manipulating the PS3 backup data. When I say “PS3 backup”, I’m not talking about a “backup” of a game, no.. I’m talking about the full PS3 hard drive backup that you can do by going to “System Settings->Backup Utility” on your XMB. That creates an encrypted directory on your FAT32 hard drive which allows you to format your PS3 and then Restore it just like it was before. I’ve reverse engineered the file format and encryption and PS3xport allows you to create new backup data from scratch, or dump existing ones, or delete specific files from a backup or do a whole lot of other things to your backup folders. This gives you total control over your /dev_hdd0 and /dev_flash2 filesystems, which will let you install homebrew on any console, even if it’s the latest OFW version. Unfortunately, just like it was 3 years ago, you wouldn’t be able to run those homebrew apps you install due to the NPDRM ECDSA signature missing. If you have your IDPS though for example, it could let you restore a backup from one PS3 to another PS3 without losing any of your data in the transfer.
So.. what’s this about “your IDPS”? yes, the backup has two sets of files, some can be decrypted right away and some can’t because they are encrypted with your IDPS (your unique ps3 device id) which is why they can’t be restored on a different ps3. If you have a CFW, you can easily get your IDPS (I’ve written a small tool to do that, released on github, but apparently MM and Webman will also give you that information) and that will give you total control over your backup data as you would be able to decrypt and reencrypt it. If you have OFW and can’t get your IDPS, then you will not be able to dump/decode all the files from your backup, but you will still be able to create a backup that can be restored on your PS3 with no limitations (this means for example that you can restore a backup from a CFW into an OFW without any issues). I was told however that someone can get IDPS from OFW consoles and in light of this release, they might release their method soon, I can’t say more than that though, but be patient and good things come to those who wait
So my release is in two parts. First, the documentation of the file format was added to theps3devwiki so any developer can understand how the backup archive files are created and can create their own tools. Reverse engineering that format took months of work and I won’t go into too much details about what had to be done to figure out the format but it was an incredibly long and difficult task to do that I had a lot of fun in doing. The second part of the release is of course the release of the ps3xport tool. The tool is quite powerful and you can do a lot of things with it, but it’s a command line only tool and I honestly just tested it on Linux, it’s not really my job at this point to make a windows build, or make a GUI around it, etc.. but I’m sure it won’t be long before others in the scene pick it up and make a nice GUI for it and release windows binaries. I’ve written a nice README file so everyone can understand how the tool works and what it can do. I remember though that 3 years ago just before I stopped working on it, I wanted to add a “AddPKG” command to it which would just ‘install’ a pkg into the backup data automatically, unfortunately, I never got to do it, but it should be easy to do. While I’m at it, I’m also releasing a pkg extraction tool which I found in an old directory (cool thing is the -p option in it, try it…) as well which is a PKG extraction tool that uses the PagedFile mechanism (see below) to allow for very fast pkg file access with very little memory usage even for huge pkg files, any dev can probably mix those two together to add the AddPKG feature to ps3xport.
On the software front, ps3xport.c will parse the commands then use the archive_* API which is in archive.c. That will contain all the functions needed to manipulate the archive files. It uses a ChainedList which is my rudimentary implementation of a GList-like ordered list and the archive API also uses PagedFile objects which are pretty cool. PagedFiles are a wrapper around a file which allows you to read/write to a file using pages (I set it to 64KB per page I think) so it limits the hard drive access. The cool thing about it is that it has encryption and hashing built in, so you can just set the encryption key or ask for the file to be hashed, and whenever you read/write, the encryption will be done transparently, and the coolest thing about it is that you can actually seek in the encrypted file and it will still work (it recalculates the required IV whenever you seek). The encryption there works on the stream, so you don’t need to write blocks of 16 bytes every time (thanks to the paging of the data) and it has a cool ‘splice’ method which allows you to copy data from one PagedFile to another easily, so you could in theory re-encrypt a file using a different key using 5 function calls (open *2, set_key*2, splice).
Download:
https://github.com/kakaroto/ps3xport
Source:
More needed info about Data Backup can bé find below
http://www.psdevwiki.com/ps3/Archive.dat
Read previous article below.
http://www.xxzer0modzxx.net/forum/index.php?threads/early-days-ps3-4-0-hen-by-kakaroto.46/
Last edited:
Previous thread
Next thread
