By default, accessing third party relational databases can be very slow if not configured properly. I recently started using PostgreSQL 9.1, an open source database, to store high volumes of data for my SAS Global Forum 2013 paper. In this example I am using an ODBC connection to connect to a database I defined in Postgres. Native SAS/ACCESS engines to databases such as Oracle or MS SQL Server will have different options and abilities. At first it was taking forever to load up data because SAS was inserting 1 row at a time into the database table. After adding a simple option my data processing was off to the races! Buffering Options The SAS INSERTBUFF and READBUFF options will improve ODBC and OLE DB libraries dramatically. By default these are set to 1 and 250 rows respectively for ODBC connections. Setting these buffer sizes tells SAS how many rows to buffer in memory before processing. Using the LIBNAME Option These options can be added to the LIBNAME statement to set the buffering sizes for all processing done on tables within the library. Ideally if you have the SAS Metadata server running, your SAS Administrator should set these options through [...]
The post SAS Administration: Process Data Faster in RDBMS by Buffering the Data in Memory appeared first on Business Intelligence Notes for SAS® BI Users. Written by Steve Overton.