Thread: deSQP Sqp archive manager Full Source

deSQP Full C-Sharp Source is a complete project for extracting or re-packing SQP archives.

Key features:

• Threads
• WPF
• Reading and writing binary files
• Packing and unpacking data with zLib
• Exporting and using functions from C++ library
• Loading libraries from resources into memory (read special notices)
• Playing XM media (read special notices)

How this thing works? There are two major parts

• Actual unpacker/packer with libraries
• File list creator

I will not pay much attention to how exactly works deSQP (there are enough comments in source), but will describe main aspects of using library, which was written in C++ by Genz.

SQPE.dll provides several functions for extracting data from SQP and packing them back. This library can’t create completely new SQP archive or add new files into existed one. You can only replace files with same names. That was done due simple reason: game client do not know anything about your files with custom names.

Part 1. deSQP.

This tool was written in C# WPF. It utilizes one special dll: SQPE.dll which exports following functions:

• sqpUnpack (IntPtr archiveName, IntPtr destFolder);
  archiveName – pointer to archive name
  destFolder – pointer to destination folder
• sqpPack (IntPtr archiveName, IntPtr sourceFolder, bool doCompact);
  archiveName – pointer to string with archive name
  sourceFolder – pointer to string with source Folder.
  doCompact - Whether archive should be compressed or not.
• GetErrorCode()
• sqpInit (IntPtr NamesFile);
  NamesFile – pointer to array of bytes with filenames, hashes etc.
• sqpUninit();
• sqpEmergencyExit (UInt32 Value);
• sqpGetFilesCount()
• sqpGetProgress()
• sqpGetFileName()

First of all, _sqpInit with NamesFile parameter should be called. This function prepares needed structures.

Code:

// Before loading SQP module, we should pass a list of available file names
 
// load file list from resources and pass pointer to dll
IntPtr filelistPtr = LoadFileList();
 
//init sqp module
sqpInit(filelistPtr);

where LoadFileList() reads data from binary file, allocates memory, saves data in allocated memory and returns pointer to this memory region.
More about binary file with filenames in section 2.

After that we can use sqpUnpack and sqpPack in order to Unpack or Pack data.
You can notice that during unpacking the “Files.db” is created inside folder with extracted data. This file stores some info about unpacked files and can be safely removed if you don’t plan to repack SQP. If Files.db have been corrupted or removed, sqpPack will return error and repacking will not be finished.

Also sqpPack has doCompact parameter which can be set to True for compressing output SQP. It is a native SQP feature so game client should work with packed archive (I’ve never tested it, so I can’t say anything else)

Part 2. SQPF creator.

As we know, SQP archive doesn’t store file names or even file paths. Each file name represented as combination of two hashes.
To properly extract data from SQP we got all available patches from game servers, extracted them (‘cuz they contain files with readable names) and make a special binary file. sqpf.dat stores full filenames, paths and filename hashes.
The structure of sqpf.dat is:

Code:

struct sqpfHeader
{
            dword FileSig; // = 53515046, "SQPF"
            dword FileCount;
}
 
struct sqpFiles
{
            ubyte FileNameLength;
            char  fileName[FileNameLength];
            ubyte Zero;
            dword HashA;
            dword HashB;
}

SQPF creator just read line-by-line plain text file

Code:

Bin\Config\DynamicGroup.xml
Bin\Config\EnergyLevelUp.xml
Bin\Config\G_configures.xml
Bin\Config\MapLinkWays.xml
Bin\Config\Setting\Color.xml

then calculate hashes and write all data into binary file.
Main functions are: BuildStormBuffer, HashString. Before calculating hashes we should init keys array by calling BuildStormBuffer(); and then, call for each filename HashString()

Code:

hashA = HashString(tempstr, HASH_NAME_A);
hashB = HashString(tempstr, HASH_NAME_B);

Functions:

Code:

        //Init hash 
        private static uint[] sStormBuffer;
        const int HASH_NAME_A = 1;
        const int HASH_NAME_B = 2;
 
        /// <summary>
        /// Init sStormBuffer for hashing
        /// </summary>
        /// <returns>Array of dwords</returns>
        private static uint[] BuildStormBuffer()
                        {
                                   uint seed = 0x100001;
                                  
                                   uint[] result = new uint[0x500];
                                  
                                   for(uint index1 = 0; index1 < 0x100; index1++)
                                   {
                                               uint index2 = index1;
                                               for(int i = 0; i < 5; i++, index2 += 0x100)
                                               {
                                                           seed = (seed * 125 + 3) % 0x2aaaab;
                                                           uint temp = (seed & 0xffff) << 16;
                                                           seed = (seed * 125 + 3) % 0x2aaaab;
 
                                                           result[index2]  = temp | (seed & 0xffff);
                                               }
                                   }
 
                                   return result;
                        }
 
        /// <summary>
        /// Creating hash for string name. Before calling this function
        /// sStormBuffer should be initialized
        /// </summary>
        /// <param name="input">Input string name</param>
        /// <param name="offset">Hash type: 1 or 2</param>
        /// <returns></returns>
        internal static uint HashString(string input, int offset)
                        {
            // magic seeds
                                   uint seed1 = 0x7fed7fed;
                                   uint seed2 = 0xeeeeeeee;
                                  
                                   foreach(char c in input)
                                   {
                                               int val = (int)char.ToUpper(c);
                                               seed1 = sStormBuffer[(offset << 8 ) + val] ^ (seed1 + seed2);
                                               seed2 = (uint)val + seed1 + seed2 + (seed2 << 5) + 3;
                                   }
                                   return seed1;
                        }
 
 
    }

As additional, “SQPF creator” pack output file with zlib.


Dependences:

• ZLibNet (zlib Home Site)
• DMex (random::entropy )
• uFMOD (ufmod.sourceforge.net)

Project already contains needed classes and dlls, so everything should be compiled without problem.
Also I’ve added comments to every function.

What should be changed (if someone would take this job):

• Read filename directly from file, not from resources
• Finish CheckSQPpack function

P.S. If you will make modifications or will use SQPE.dll in your own projects, please, save proper credits and notify me about your work.
P.P.S. Also, please post your suggestion or maybe tips about improving etc…

Author: Dwar & Genz

Compiled project: https://progamercity.net/game-files/...-unpacker.html

License

This work is licensed under the Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License.
To view a copy of this license, visit Creative Commons — Attribution-NonCommercial-ShareAlike 3.0 Unported — CC BY-NC-SA 3.0

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

• Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
• Neither the name of the ProGamerCity, PGC, nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND ONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.