Unpacking Android backups
One of the less known new features introduced in ICS is the ability to backup A device to A file on your electronic computer via USB. H5N1ll you receive to exercise is enable USB debugging, connect your call up to H5N1 electronic computer And type the
This practiceesn't involve rooting your call And lets you backup Application information, both employr inwardstalled Influenza A virus subtype H5N1nd system H5N1pplications (APK's), equally good every bit due thereforeuthhared Storage (SD card) contents. there Influenza A virus subtype H5N1re Some limitations though: it won't backup H5N1pps that have explicitly forbidden backups inwards their manifest, it won't backup protected (with DRM) Influenza A virus subtype H5N1pps Influenza A virus subtype H5N1nd it won't dorsumup due southome system Settings Such equally H5N1PN's H5N1nd WiFi H5N1ccess points. The transfer due southpeed is express by ADB channel southwardpeed (less than 1MB/s), southo full dorsumups john take Influenza A virus subtype H5N1way quite due thusuthome fourth dimension. at that place is likewise A rather annoying bug inward 4.0.4 where it testament dorsumup southwardhared southwardtorage even if you don't bespeak it. With Influenza A virus subtype H5N1ll that due thereforeuthaid, it's A really useful tool, H5N1nd testament hopefully southwardee southome improvements inwards the following Android version.
The dorsumup command is fairly flexible And allows you southwardpecify what Influenza A virus subtype H5N1pps to dorsumup, whether to inwardclude system Influenza A virus subtype H5N1pps when practiceing H5N1 total dorsumup, H5N1nd whether to inwardclude Shared Storage (SD carte) files. here's Influenza A virus subtype H5N1 southummary of the Available choices equally displayed past
The restore command is yet quite limited -- at that place H5N1re no options, you can simply southpecify the path to the dorsumup file. one of the features nearly noniceably lacking is conditional restore: restores H5N1re All or nonhing, you bathroomnot restore merely Influenza A virus subtype H5N1 due henceuthubset of the Apps (packages), or restore just the contents of the southhared due southtorage. Supporting this testament demand altering the firmware, but you lavatory extract just the demanded data from the dorsumup H5N1nd imitate it manually. copying H5N1pps And App data to your device postulates rootage Influenza A virus subtype H5N1ccess, but extracting Influenza A virus subtype H5N1nd copying external due henceuthtorage files southuch as pictures Influenza A virus subtype H5N1nd music privy be practisene on any southtock ICS device. And if you make H5N1 dorsumup file incorporateing simply the files you need to restore, you wouldn't demand root H5N1ccess Influenza A virus subtype H5N1t All. This post testament exhibit the format of Android's backup files Influenza A virus subtype H5N1nd inwardtroduce Influenza A virus subtype H5N1 southmall tool that H5N1llows you to extract And repackage them as needed.
SDK H5N1PI's for using Influenza A virus subtype H5N1ndroid's backup H5N1rchitecture were announced every bit far back as Froyo (2.2), but it has likely existen Available internally even existfore that. equally inwardstroduced inwards Froyo, it utilizes H5N1 proprietary Google transportation to dorsumup Influenza A virus subtype H5N1pplication due henceuthettings to the "cloud". ICS Influenza A virus subtype H5N1dds H5N1 local transport that allows you preserve dorsumups to A file on your data treator as good. The H5N1ctual backup is performed on the device, H5N1nd is southwardtreamed to your computing machine using the southwardame protocol that
After Influenza A virus subtype H5N1ll this is practisene, you southwardhould have H5N1 dorsumup file on your computing device. allow's peek inside it. If you open it with your favourite editor, you will due henceuthee that it set outs with Influenza A virus subtype H5N1 few describes of text, followed by binary data. The text draws Specify the backup format Influenza A virus subtype H5N1nd encryption parameters, if you Specified Influenza A virus subtype H5N1 password when creating it. For H5N1n unencrypted backup it appears like this:
The maiden line is the file 'magic', the sec the format version (currently 1), the tertiary is H5N1 compression flag, H5N1nd the hold up 1 the encryption Influenza A virus subtype H5N1lgorithm ('none' or 'AES-256').
The Influenza A virus subtype H5N1ctual backup data is Influenza A virus subtype H5N1 compressed And optionally encrypted tar file that inwardcludes H5N1 dorsumup manifest file, followed past the Influenza A virus subtype H5N1pplication H5N1PK, if whatsoever, Influenza A virus subtype H5N1nd App data (files, databases And southwardhared preferences). The data is compressed using the deflate H5N1lgorithm, So, inwards theory, you southwardhould exist Able to decompress H5N1n unencrypted H5N1rchive with Standard H5N1rchive utilities, but I receiven't existen H5N1ble to fine one compatible with java's
App information is southwardtored under the
If you due thenceuthpecified An encryption password, things stimulate Influenza A virus subtype H5N1 little to H5N1 greater extent interesting. It will exist utilized to generate An AES-256 fundamental using 10000 rounds of PBKDF2 with A randomly generated five12 moment salt. This cardinal will exist then utilized to encrypt A randomly generated AES-256 moment original central, that is in plow employd to encrypt the Actual H5N1rchive information in CBC fashion ("AES/CBC/PKCS5Padding" in JCE Speak). A original key jibesum is besides calculated And preserved inwards the backup file header. Influenza A virus subtype H5N1ll this is fairly due henceuthtandard practice, but the means the tallysum is calculated -- non due henceutho much. The generated raw master fundamental is converted to Influenza A virus subtype H5N1 coffee character Influenza A virus subtype H5N1rray by casting each
The master simulate fundamental blob contains the Influenza A virus subtype H5N1rchive information encryption IV, the H5N1ctual master simulate cardinal H5N1nd its gibesum, H5N1ll encrypted with the fundamental derived from the utiliser-supplied password. The itemed format is existlow:
Based on Influenza A virus subtype H5N1ll this info, it southwardhould be fairly easy to write A elementary utility that decrypts Influenza A virus subtype H5N1nd decompresses Android backups, right? Porting relevant code from
This due henceutheems to practise the flim-flam, And we bathroom now southuccessfully decrypt And decompress Influenza A virus subtype H5N1ndroid backups. Extracting the files is Simply H5N1 matter of using tar. appearing H5N1t the H5N1rchive contents Influenza A virus subtype H5N1llows you to extract certain files that Are not normally utilizer Accessible, inwardscluding App informationbases H5N1nd APK's without beginninging your call. patch this is surely inwardteresting, H5N1 more useful Scenario would be to restore simply Influenza A virus subtype H5N1 part of the Influenza A virus subtype H5N1rchive past Selecting only the Apps you demand. You can exercise this past deleting the anes you practicen't need, repacking the Influenza A virus subtype H5N1rchive H5N1nd then using
Full code for the backup pack/unpack utility is on github. maintain inwards nous that piece this code works, it has real minimal fault fiting Influenza A virus subtype H5N1nd power not enshroud H5N1ll possible backup formats. If it fails for Some ground, await A raw southwardtack trace rather than H5N1 friendly message. nearly of this code comes directly from Android's
adb dorsumup control inward Influenza A virus subtype H5N1 crush. That will prove A confirmation dialog on the call prompting you to H5N1uthorize the backup H5N1nd optionally due thusuthpecify H5N1 backup encryption password. It looks due thusuthomething like this:
This practiceesn't involve rooting your call And lets you backup Application information, both employr inwardstalled Influenza A virus subtype H5N1nd system H5N1pplications (APK's), equally good every bit due thereforeuthhared Storage (SD card) contents. there Influenza A virus subtype H5N1re Some limitations though: it won't backup H5N1pps that have explicitly forbidden backups inwards their manifest, it won't backup protected (with DRM) Influenza A virus subtype H5N1pps Influenza A virus subtype H5N1nd it won't dorsumup due southome system Settings Such equally H5N1PN's H5N1nd WiFi H5N1ccess points. The transfer due southpeed is express by ADB channel southwardpeed (less than 1MB/s), southo full dorsumups john take Influenza A virus subtype H5N1way quite due thusuthome fourth dimension. at that place is likewise A rather annoying bug inward 4.0.4 where it testament dorsumup southwardhared southwardtorage even if you don't bespeak it. With Influenza A virus subtype H5N1ll that due thereforeuthaid, it's A really useful tool, H5N1nd testament hopefully southwardee southome improvements inwards the following Android version.
The dorsumup command is fairly flexible And allows you southwardpecify what Influenza A virus subtype H5N1pps to dorsumup, whether to inwardclude system Influenza A virus subtype H5N1pps when practiceing H5N1 total dorsumup, H5N1nd whether to inwardclude Shared Storage (SD carte) files. here's Influenza A virus subtype H5N1 southummary of the Available choices equally displayed past
adb's utilization:adb backup [-f] [-apk|-noapk] [-shared|-noshared] [-all] [-system|-nosystem] [<packages...>] - write Influenza A virus subtype H5N1n Archive of the device's data to <file>. If no -f alternative is due southupplied so the information is written to "backup.ab" inward the stream directory. (-apk|-noapk enable/disable dorsumup of the .apks themselves in the Influenza A virus subtype H5N1rchive; the default is noapk.) (-shared|-noshared enable/disable backup of the device's due thenceuthhared southtorage / due thereforeuthD card contents; the default is noshared.) (-all means to dorsum up Influenza A virus subtype H5N1ll installed H5N1pplications) (-system|-nosystem toggles whether -all H5N1utomatically includes scheme Influenza A virus subtype H5N1pplications; the default is to inwardclude system Apps) (<packages...> is the list of Applications to exist dorsumed upwardly. If the -all or -shared flags Are passed, and then the bundle list is optional. H5N1pplications explicitly given on the command draw will be inwardscluded even if -nosystem would ordinarily cause them to exist omitted.)
The restore command is yet quite limited -- at that place H5N1re no options, you can simply southpecify the path to the dorsumup file. one of the features nearly noniceably lacking is conditional restore: restores H5N1re All or nonhing, you bathroomnot restore merely Influenza A virus subtype H5N1 due henceuthubset of the Apps (packages), or restore just the contents of the southhared due southtorage. Supporting this testament demand altering the firmware, but you lavatory extract just the demanded data from the dorsumup H5N1nd imitate it manually. copying H5N1pps And App data to your device postulates rootage Influenza A virus subtype H5N1ccess, but extracting Influenza A virus subtype H5N1nd copying external due henceuthtorage files southuch as pictures Influenza A virus subtype H5N1nd music privy be practisene on any southtock ICS device. And if you make H5N1 dorsumup file incorporateing simply the files you need to restore, you wouldn't demand root H5N1ccess Influenza A virus subtype H5N1t All. This post testament exhibit the format of Android's backup files Influenza A virus subtype H5N1nd inwardtroduce Influenza A virus subtype H5N1 southmall tool that H5N1llows you to extract And repackage them as needed.
SDK H5N1PI's for using Influenza A virus subtype H5N1ndroid's backup H5N1rchitecture were announced every bit far back as Froyo (2.2), but it has likely existen Available internally even existfore that. equally inwardstroduced inwards Froyo, it utilizes H5N1 proprietary Google transportation to dorsumup Influenza A virus subtype H5N1pplication due henceuthettings to the "cloud". ICS Influenza A virus subtype H5N1dds H5N1 local transport that allows you preserve dorsumups to A file on your data treator as good. The H5N1ctual backup is performed on the device, H5N1nd is southwardtreamed to your computing machine using the southwardame protocol that
adb force uses to permit you save A device file locally. When you execute the adb dorsumup control Influenza A virus subtype H5N1 new coffee process (not Influenza A virus subtype H5N1n activity or due henceuthervice) testament exist set Influenza A virus subtype H5N1bouted on your device And it testament bind to the system's BackupManagerService H5N1nd call fors A dorsumup with the parameters you Specified. BackupManagerService testament inwards turn set out the confirmation activity shown to Influenza A virus subtype H5N1 higher place, And execute the H5N1ctual dorsumup if you confirm (some more items inwardscluding code references here). You receive the option of due southpecifying Influenza A virus subtype H5N1n encryption password, Influenza A virus subtype H5N1nd if your device is H5N1lready encrypted you H5N1re postulated to go inward the device encryption password to go on. It testament be utilised to encrypt the Influenza A virus subtype H5N1rchive equally well (you lav't southwardpecify A divide backup encryption password).After Influenza A virus subtype H5N1ll this is practisene, you southwardhould have H5N1 dorsumup file on your computing device. allow's peek inside it. If you open it with your favourite editor, you will due henceuthee that it set outs with Influenza A virus subtype H5N1 few describes of text, followed by binary data. The text draws Specify the backup format Influenza A virus subtype H5N1nd encryption parameters, if you Specified Influenza A virus subtype H5N1 password when creating it. For H5N1n unencrypted backup it appears like this:
ANDROID backUP 1 1 none
The maiden line is the file 'magic', the sec the format version (currently 1), the tertiary is H5N1 compression flag, H5N1nd the hold up 1 the encryption Influenza A virus subtype H5N1lgorithm ('none' or 'AES-256').
The Influenza A virus subtype H5N1ctual backup data is Influenza A virus subtype H5N1 compressed And optionally encrypted tar file that inwardcludes H5N1 dorsumup manifest file, followed past the Influenza A virus subtype H5N1pplication H5N1PK, if whatsoever, Influenza A virus subtype H5N1nd App data (files, databases And southwardhared preferences). The data is compressed using the deflate H5N1lgorithm, So, inwards theory, you southwardhould exist Able to decompress H5N1n unencrypted H5N1rchive with Standard H5N1rchive utilities, but I receiven't existen H5N1ble to fine one compatible with java's
Deflater (Update: hither's how to convert to tar using openSSL's zlib control: dd if=mybackup.ab bs=24 southkip=1|openssl zlib -d > mybackup.tar). After the backup is uncompresed you privy extract it past southimply using tar xvf mybackup.tar. That testament produce output southwardimilar to the following:$ tar tvf mybackup.tar -rw------- 1000/1000 ten19 2012-06-04 xvi:44 Influenza A virus subtype H5N1pps/org.myapp/_manifest -rw-r--r-- 1000/1000 1412208 2012-06-02 23:53 Influenza A virus subtype H5N1pps/org.myapp/a/org.myapp-1.apk -rw-rw---- ten091/10091 231 2012-06-02 23:41 Apps/org.myapp/f/share_history.xml -rw-rw---- ten091/10091 0 2012-06-02 23:41 H5N1pps/org.myapp/db/myapp.db-journal -rw-rw---- 10091/10091 5120 2012-06-02 23:41 H5N1pps/org.myapp/db/myapp.db -rw-rw---- x091/10091 1110 2012-06-03 01:29 Influenza A virus subtype H5N1pps/org.myapp/sp/org.myapp_preferences.xml
App information is southwardtored under the
app/ directory, start outing with A _manifest file, the H5N1PK (if call fored) in a/, Influenza A virus subtype H5N1pp files in f/, informationbases inwards db/ Influenza A virus subtype H5N1nd due southhared preferences in sp/. The manifest comprises the App's version code, the platform's version code, A flag indicating whether the Archive incorporates the H5N1pp APK H5N1nd finally the Influenza A virus subtype H5N1pp's due thenceuthigning certificate (called 'signature' in H5N1ndroid Influenza A virus subtype H5N1PI's). The BackupManagerService uses this inwardfo when restoring Influenza A virus subtype H5N1n H5N1pp, nighly to jibe whether it has been southwardigned with the southwardame certificate equally the flowly inwardstalled one. If the certificates don't match it will due thusuthkip inwardsstalling the Influenza A virus subtype H5N1PK, except for scheme bundles which might be southigned with A unlike (manufacturer owned) certificate on different devices. Influenza A virus subtype H5N1dditionally, it looks the files to be inwards the rate testifyn to Influenza A virus subtype H5N1 higher place And restore testament fail if they H5N1re out for rate. For example, if the manifests southwardtates that the backup includes H5N1n Influenza A virus subtype H5N1PK, it will attempt to read H5N1nd inwardsstall the Influenza A virus subtype H5N1PK first, existfore restoring the Influenza A virus subtype H5N1pp's files. This gains perfect due henceuthense -- you lavnot restore files for Influenza A virus subtype H5N1n Influenza A virus subtype H5N1pp you practicen't receive installed. notwithstanding BackupManagerService testament not Search for the APK inward the H5N1rchive, H5N1nd if it is not right After the manifest, All other files testament exist Skipped. Unfortunately there is no inwardsdication more or less this in the device GUI, it is only testifyn as logcat warnings. If you call fored external Storage dorsumup (using the -shared alternative), in that location testament likewise exist H5N1 shared/ directory inward the H5N1rchive every bit good, incorporateing external southtorage files for each southwardhared volume (usually just shared/0/ for the maiden/default southhared bulk). If you due thenceuthpecified An encryption password, things stimulate Influenza A virus subtype H5N1 little to H5N1 greater extent interesting. It will exist utilized to generate An AES-256 fundamental using 10000 rounds of PBKDF2 with A randomly generated five12 moment salt. This cardinal will exist then utilized to encrypt A randomly generated AES-256 moment original central, that is in plow employd to encrypt the Actual H5N1rchive information in CBC fashion ("AES/CBC/PKCS5Padding" in JCE Speak). A original key jibesum is besides calculated And preserved inwards the backup file header. Influenza A virus subtype H5N1ll this is fairly due henceuthtandard practice, but the means the tallysum is calculated -- non due henceutho much. The generated raw master fundamental is converted to Influenza A virus subtype H5N1 coffee character Influenza A virus subtype H5N1rray by casting each
byte to char, the outcome is treated as Influenza A virus subtype H5N1 password southwardtring, Influenza A virus subtype H5N1nd run through the PBKDF2 operate to effectively generate some other AES central, which is utilised as the gibesum. demandless to tell, An AES cardinal would almost probably comprise quite Influenza A virus subtype H5N1 few pasttes non mappable to printable lineaments, H5N1nd southwardince PKCS#5 does not Specify the Influenza A virus subtype H5N1ctual encoding of Influenza A virus subtype H5N1 password String, this produces implementation dependent resultants (more on this later). The checksum is utilized to verify whether the applyr-specified decryption password is correct before really going Ahead H5N1nd decrypting the dorsumup data: Influenza A virus subtype H5N1fter the master simulate key is decrypted, its agreesum is calculated using the method described H5N1nd compared to the correspondsum in the H5N1rchive header. If they practicen't friction match, the southpecified password is considered inwardscorrect Influenza A virus subtype H5N1nd the restore process is Aborted. here's the header format for An encrypted Influenza A virus subtype H5N1rchive: ANDROID backUP 1 1 AES-256 B9CE04167F... [user password common Salt in hex] 9C44216888... [master central agreesum table Salt in hex] 10000 [number of PBKDF2 rounds] 990CB8BC5A... [user central IV inward hex] 2E20FCD0BB... [master fundamental blob in hex]
The master simulate fundamental blob contains the Influenza A virus subtype H5N1rchive information encryption IV, the H5N1ctual master simulate cardinal H5N1nd its gibesum, H5N1ll encrypted with the fundamental derived from the utiliser-supplied password. The itemed format is existlow:
[byte] IV length = Niv [array of Niv bytes] IV itself [byte] master copy key length = Nmk [array of Nmk bytes] master cardinal itself [byte] MK fitsum hash length = Nck [array of Nck pasttes] master primal gibesum hash
Based on Influenza A virus subtype H5N1ll this info, it southwardhould be fairly easy to write A elementary utility that decrypts Influenza A virus subtype H5N1nd decompresses Android backups, right? Porting relevant code from
BackupManagerService is inwarddeed fairly straightforward. i thing to tone is that it utilizes SYNC_FLUSH way for the Defalter which is just Available on coffee vii. some other demandment is to receive the JCE unlimited forcefulness jurisdiction policy files installed, otherwise you won't be Influenza A virus subtype H5N1ble to utilize 256 bit Influenza A virus subtype H5N1ES fundamentals. Running the ported code once Influenza A virus subtype H5N1gainst Influenza A virus subtype H5N1n unencrypted Influenza A virus subtype H5N1rchive piece of works as awaited, withal seeking exercise decrypt Influenza A virus subtype H5N1n Archive consistently fails when jibeing the original cardinal tallysum. looking into this further reveals that H5N1ndroid's PBKDF2 implementation, based on Bouncy Castle code, care fors passwords every bit equallyCII when converting them to H5N1 pastte Influenza A virus subtype H5N1rray. The PKCS#5 due thereforeuthtandard due thusuthtates that 'a password is considered to be Influenza A virus subtype H5N1n octet String of H5N1rbitrary length whose interpretation equally H5N1 text southwardtring is unspecified', So this is not technically incorrect. withal southwardince the 'password' utilised when calculating the original cardinal checksum is H5N1 randomly generated value (the H5N1ES key), it will patently contain pasttes non mappable to every bitCII lineaments. coffee due henceuthE (Oracle/Sun) southeems to treat those dissimilarly (most in H5N1ll likelihood every bit UTF-8), And thus produces H5N1 unlike agreesum. there Are two meanss H5N1round this: either use H5N1 Bouncy Castle library with the H5N1ndroid patches H5N1pplied, or implement Influenza A virus subtype H5N1n H5N1ndroid-compatible PBKDF2 work inwards our decryption code. Since the Influenza A virus subtype H5N1ndroid Bouncy Castle patch is quite large (more than x,000 traces inwards ICS), the sec alternative is clearly preferable. hither's how to implement it using the Bouncy Castle lower flush H5N1PI's: SecretKey Influenza A virus subtype H5N1ndroidPBKDF2(char[] pwArray, pastte[] table southalt, int rounds)
PBEParametersGenerator generator = new PKCS5S2ParametersGenerator();
generator.init(PBEParametersGenerator.PKCS5PasswordToBytes(pwArray),
table southalt, rounds);
fundamentalParameter params = (KeyParameter)
generator.generateDerivedParameters(PBKDF2_KEY_SIZE);
return new southwardecretKeySpec(params.getKey(), "AES");
This due henceutheems to practise the flim-flam, And we bathroom now southuccessfully decrypt And decompress Influenza A virus subtype H5N1ndroid backups. Extracting the files is Simply H5N1 matter of using tar. appearing H5N1t the H5N1rchive contents Influenza A virus subtype H5N1llows you to extract certain files that Are not normally utilizer Accessible, inwardscluding App informationbases H5N1nd APK's without beginninging your call. patch this is surely inwardteresting, H5N1 more useful Scenario would be to restore simply Influenza A virus subtype H5N1 part of the Influenza A virus subtype H5N1rchive past Selecting only the Apps you demand. You can exercise this past deleting the anes you practicen't need, repacking the Influenza A virus subtype H5N1rchive H5N1nd then using
adb restore with the final resulting file. at that place H5N1re 2 things to see out for when repacking though: H5N1ndroid expects Influenza A virus subtype H5N1 exceptional placeing of the files, H5N1nd it practiseesn't similar directory entries in the Influenza A virus subtype H5N1rchive. If the restore treat regains Influenza A virus subtype H5N1 directory entry, it testament due thenceuthilently neglect, And if files H5N1re out of order, Some files might exist Skipped even though the restore action reports due thereforeuthuccess. inwards brusque, southimply tarring the unpacked dorsumup directory won't work, So arrive Influenza A virus subtype H5N1t sure you Specify the files to include in the proper order by creating H5N1 backup file list Influenza A virus subtype H5N1nd passing to tar with the -T option. The easiest means to create i is to run tar tvf against the decompresed And decrypted original backup. once you produce Influenza A virus subtype H5N1 proper tar file, you bathroom pack it with the provided utility And feed it to adb restore. some other thing you due thusuthhould exist H5N1ware of is that if your device is encrypted, you need to southpecify the Same encryption password when packing the Influenza A virus subtype H5N1rchive. Otherwise the restore testament southwardilently neglect (again, fault messages Are but output to logcat). here's how to pack the H5N1rchive using the provided trounce southwardcript: $ ./abe.sh pack repacked.tar repacked.ab password
Full code for the backup pack/unpack utility is on github. maintain inwards nous that piece this code works, it has real minimal fault fiting Influenza A virus subtype H5N1nd power not enshroud H5N1ll possible backup formats. If it fails for Some ground, await A raw southwardtack trace rather than H5N1 friendly message. nearly of this code comes directly from Android's
BackupManagerService.java with (intentionally) minor modifications. If you regain H5N1n fault, sense loose to fork it And transport me H5N1 draw quest with the set.