New version of spssio32.dll generates huge SAV files
Hi,
We recently implemented the latest version of the spssio32.dll in order to resolve the 255 character limit that was imposed on text fields in previous versions.
However, we are now noticing that the dll is generating huge files compared to the previous version. Files that were previously approx. 1MB in size are now over a 100MB is size even though the data is the same. We have some files that are now over 4GB in size!
Our initial thoughts are that the file is now allocation 32K characters for EVERY field rather than using dynamic sizing but have not been able to confirm this.
This is an urgent issue for us so any help you can provide would be very useful.
Regards
Bhavin
We may need more information, but you can control the width of string fields. Although the max is now 32K, you set the width in your code.
Also, if you are in Unicode mode, characters will require more bytes if they are not plain 7- bit ascii. Again, whether or not you are in Unicode mode is under your control.
I wonder also whether you have compression turned off.
Regards,
Jon Peck
Thanks for the reply Jon.
We are currently setting the width but is there any way to do this dynamically via the dll?
Specifically, every string variable is different for us and rather than setting a static value for all string variables, is there a way to let the DLL decide how wide to make the column based on the data that is being passed?
We use the spssSetVarPrintFormat to set the width of the variable but I noticed that there is also a spssSetVarColumnWidth that allows a special value of 0 to be set. Should we be using this instead.
I have provided a code snippet that illustrates how we are calling the DLL below. Any advice you can provide would be useful.
Regards
Bhavin
// Create Variable
int errCode = spss.spssSetVarName(handle, varName, varType);
if (errCode != Spss.SPSS_OK)
{
throw
new Exception(string.Format(
"Unable to set variable name: {0}\r\nError: {1}", varName, errCode));
}
// Set Variable Label errCode = spss.spssSetVarLabel(handle, varName, varLabel);
if (errCode != Spss.SPSS_OK && errCode != Spss.SPSS_EXC_LEN120)
{
throw
new Exception(string.Format(
"Unable to set variable label for variable: {0}\r\nError: {1}", varName, errCode));
} errCode = spss.spssSetVarPrintFormat(handle, varName, varPrintType, varPrintDec, varPrintWidth);
if (errCode != Spss.SPSS_OK)
{
throw
new Exception(string.Format(
"Unable to format variable: {0}\r\nError: {1}", varName, errCode));
} The varPrintWidth is what we are setting to
public
const Int32 SPSS_MAX_LONGSTRING = 32767;
There is no way to set the width dynamically, because all the data would have to be passed first in order to know how wide to make the fields. You might try opening the file in Statistics after creation is complete and run ALTER TYPE a to z (AMIN). That would show you what the minimum field widths could be and how big the sav file is afterwards. TheColumnWidth parameter is not relevant here. It is just the width setting for Data Editor displays. The varType in your create code above is the only thing that could affect size.
Be sure that you have called spssSetCompression with a switch value of 1. The default is uncompressed.