We are currently exporting a large amount of data (potentially millions of rows) to Excel and would like to know if there is an easy way to split the files with minimal coding as we have over 100 tables to export. The files would need a unique name each - eg myTable1, myTable2 etc.

Thank you

This message has been edited. Last edited by: FP Mod Chuck,

The best thing I can think of is to have a where clause based on a range of parameters that can be provided at run time to limit the output. The hold file name can also be a parameter.

A second thought would be to use Table of Contents which would create a separate tab for each value of the table of contents field.

Thanks Chuck. I have since been considering using a WHERE clause but was hoping there might be some native functionality to split based on 'x' number of records as I don't know how big the tables are and they grow over time.

Hi iDuncanW

I wish there were RECORDLIMIT between x and y functionality but there isn't. If you can use a date range for the WHERE it might help you limit the records.

something like this?

-set &blocksize=50000;  /* number of row per output file */
table file hold1
list *      /* add a column of sequence numbers, list/i6 */
on table hold as hold2
end
-run

-set &files = (&lines + &blocksize -1)/&blocksize;

-repeat :loop: for &i from 1 to &files 

-set &hi = &i * &blocksize;
-set &lo = &hi + 1 - &blocksize;

table file hold2
print {the list of fields}
where list from &lo to &hi
on table hold as 'xls&i' format xlsx
end
-run
-:loop:

Thank you, I've been away from WebFocus for 9 years so might take me a while to get my head around that one.

Will give it a try.

IDuncanW

I got curious about this code and have made it work. Thanks j.gross that is a great technique!

I had to make sure that the IBIF_excelservurl setting in the client administration console under Client Settings / General was blanked out or I got an error.

Here is the updated code.


APP HOLD baseapp
-DEFAULTH &blocksize='',&files='',&hi='',&lo='';
-SET &blocksize=50000;

TABLE FILE WF_RETAIL_CUSTOMER
LIST *
ON TABLE HOLD AS HOLD1
END
-RUN

-SET &files = (&LINES + &blocksize -1)/&blocksize;

-REPEAT XLOOP FOR &I FROM 1 TO &files

-SET &hi = &I * &blocksize;
-SET &lo = &hi + 1 - &blocksize;

TABLE FILE HOLD1
PRINT *
WHERE E01 FROM &lo TO &hi
ON TABLE HOLD AS 'xls&I' FORMAT XLSX
END
-RUN
-XLOOP


All you should have to do is replace the WF_RETAIL_CUSTOMER with whatever MASTER file you want to use. I did a PRINT * against the HOLD1 file which includes the LIST sequence field. If you don't want that then you will have to use PRINT with the individual field names.

In my case it created 8 xlsx files based on the number of records in the WF_RETAIL_CUSTOMER table. You can increase the value of &blocksize to have more records in each xlsx file.

Let us know how it goes.

I would note that this could be a slow process. If there are two million records, it will read through allof them, 40 times, to produce 40 Excel hold files.

If you initially transfer HOLD the data as a two-segment XFocus file, sorted on the (computed) "block" number as root segment key, and on LIST as the key of the child segment, then the 40 'table ... hold format xlsx' operations (filtering with where block eq &i) will each read just the relevant rows (rather then plowing thru all 2mm records).

Gotta run; I'll post pseudo code shortly.

As before, untested, lowercased pseudo-code.

-set &blocksize=50000; 
define file hold1
  recs/i9=1;
  recno/i9=recno+1;
  blockno/i5=(recno+&blocksize-1)/&blocksize;
end
table file hold1
sum recs 
  by blockno
print {a list of columns}
  by blockno
  by recno
on table hold as hold2 format xfocus index blockno
end
-run
-set &files = (&lines + &blocksize -1)/&blocksize;

-repeat :loop: for &i from 1 to &files 
table file hold2
print {a list of columns}
  by recno noprint
where blockno eq &i ;
on table hold as 'xls&i' format xlsx
end
-run
-:loop:

This message has been edited. Last edited by: j.gross,