Hi Szabolcs:
Previous reply did not have any examples mostly because you are asking for a fast LibraryLink solution. What comes to mind is a a Mathematica equivalent to csvkit (python project). Expanding on the HSQL JDBC suggestions here is some code.
The following csv file is not large but it is what I could find from quick google search.
http://ourairports.com/data/airports.csv
I downloaded the file and save locally.
csvfile = "C:\\stuff\\airports.csv";
namedDir =
ToUpperCase[
StringReplace[Last[FileNameSplit[csvfile]], ".csv" -> ""]];
dir = CreateDirectory[namedDir];
fileDir = CreateDirectory[FileNameJoin[{dir, "file"}]];
CopyFile[csvfile,
FileNameJoin[{fileDir, Last[FileNameSplit[csvfile]]}]];
dbfile = Export[
FileNameJoin[{dir,
StringJoin[
StringReplace[Last[FileNameScsvfile]], ".csv" -> ""],
".hsqldb"]}], ExportString["", "Text"], "Text"];
Needs["DatabaseLink`"]
dbconn = OpenSQLConnection[JDBC["HSQL(Standalone)", dbfile]]
SQLExecute[dbconn, "CREATE TEXT TABLE PUBLIC.AIRPORTS(
ID VARCHAR(10),
IDENT VARCHAR(8),
TYPE VARCHAR(14),
NAME VARCHAR(256),
LATITUDE_DEG VARCHAR(16),
LONGITUDE_DEG VARCHAR(16),
ELEVATION_FT VARCHAR(8),
CONTINENT VARCHAR(4),
ISO_COUNTRY VARCHAR(4),
ISO_REGION VARCHAR(4),
MUNICIPALITY VARCHAR(8),
SCHEDULED_SERVICE VARCHAR(4),
GPS_CODE VARCHAR(8),
IATA_CODE VARCHAR(4),
LOCAL_CODE VARCHAR(10),
HOME_LINK VARCHAR(256),
WIKIPEDIA_LINK VARCHAR(256),
KEYWORDS VARCHAR(256)
)"]
SQLExecute[dbconn, "SET TABLE PUBLIC.AIRPORTS SOURCE \
\"./file/airports.csv;ignore_first=true;all_quoted=true;encoding=UTF-\
8\";"]
The schema for sql table AIRPORTS was deduced through inspection. But I would believe that a Mathematica csv Importer would need to do some auto-inspection to deduce the datatypes. Unfortunately the csv file chosen did not have any datetime values. If successful then do:
SQLExecute[dbconn, "SELECT * FROM PUBLIC.AIRPORTS"]
When done
CloseSQLConnection[dbconn]
