apache spark - Scala - Expanding an argument list in a pattern matching expression -

-

i'm new scala , trying use interface spark. i'm running problem making generic csv dataframe function. example, i've got csv 50 fields, first of task, name, , id. can following work:

val reader = new csvreader(new stringreader(txt))  reader.readall().map(_ match {   case array(task, name, id, _*) => row(task, name, id)   case unexpectedarrayform =>     throw new runtimeexception("record did not have correct number of fields: "+ unexpectedarrayform.mkstring(",")) })

however, i'd rather not have hard code number of fields needed create spark row. tried this:

val reader = new csvreader(new stringreader(txt))  reader.readall().map(_ match {   case array(args @ _*) => row(args)   case unexpectedarrayform =>     throw new runtimeexception("record did not have correct number of fields: "+ unexpectedarrayform.mkstring(",")) })

but creates row object single element. how can make expand args in row(args) if have array of n elements i'll row n elements?

this should trick:

val reader = new csvreader(new stringreader(txt))  reader.readall().map(_ match {   case a: array[string] => row(a:_*)   case unexpectedarrayform =>     throw new runtimeexception("record did not have correct number of fields: "+ unexpectedarrayform.mkstring(",")) })

edited correct omission of array type


Comments

Post a Comment

Popular posts from this blog

node.js - NodeJS remote terminal to Dropbear OpenWRT-Server -

-
i wandering whether there valid solution have remote terminal openwrt-box out of nodejs-app? connecting terminal works: ssh -i ~/.mykeys/id_rsa root@192.168.178.39 busybox v1.23.2 (2015-04-22 23:25:48 utc) built-in shell (ash) root@openwrt:~# the only interactive ssh solution nodejs doesn't interactive part described in readme.md following: var client = require('ssh2').client; var conn = new client(); conn.on('ready', function() { console.log('client :: ready'); conn.shell(function(err, stream) { if (err) throw err; stream.on('close', function() { console.log('stream :: close'); conn.end(); }).on('data', function(data) { console.log('stdout: ' + data); }).stderr.on('data', function(data) { console.log('stderr: ' + data); }); stream.end('ls -l\nexit\n'); }); }).connect({ host: '192.168.100.100', port: 22, username...
Read more

python - jinja2: TemplateSyntaxError: expected token ',', got 'string' -

-
i new jinja2 , having issue using python regular expression (re). in following code bold lines have error string in them. {% block content %} <div class="container"> {% l in lines %} {% if re.search(r"error", l) %} {# <<< throws error #} <b> {{ l }} </b> {% else %} {{ l }} <hr> {% endif %} {% endfor %} </div> {% endblock %} the re.search above throws following error: jinja2.exceptions.templatesyntaxerror templatesyntaxerror: expected token ',', got 'string' raw python code not supported in jinja2 template syntax. {% if re.search(r"error", l) %} replace line with {% if "error" in l %} can fix problem. if logical condition more complicated, should consider defining own custom filters (which can call python code) or complicated things in view layer. go check global namespace .
Read more

Qt4: how to send QString inside a struct via QSharedMemory -

-
i have struct struct control_data{ int column_number; qstring cell; }; i need send thread of qsharememory. read can't because qstring contains pointers inside. other ways? you have serialize struct byte array. can convert qstring const char* this: mystring.tostdstring().c_str(); but serializing qstring should work. the first step serialize struct qdatastream using qt, example here . then once struct can read , written can pass shared memory. a complete example of using qsharedmemory can found here . here relevant code: // first, test whether shared memory segment attached process. // if so, detach if (sharedmem.isattached()) { sharedmem.detach(); } ... qbuffer buffer; buffer.open( qbuffer::readwrite ); qdatastream out( &buffer ); out << youstruct; int size = buffer.size(); // size of int + size of qstring in bytes if ( !sharedmem.create( size ) ) { return; } // write shared memory sharedmem.lock(); char *to = (char*...
Read more