java - Getting Exception: "Read a negative frame size (-2147418110)!" While running hive queries multithreading way -

-

i've integrate hive, hdfs, sqoop functionality java & working fine. executing in sequential manner.

so i'm trying implement multithreading improve performance.

this sample template of workflow.

1) main java class --> calling 1 dao class function:(in method i've write several hive opertation join on table, group etc. inserting matched result antoher tables)

2) in dao class there lots steps(loading data hdfs, filter out records & insert table etc.) figured out steps independent of each other. put steps 4 different threads, step execute in parallel.

3) in 4 threads i'm calling functions of other class, again performaing different hive queries. i'm performaning operation on hive parallely.

so here quesion

1) can execute hive queries in multithreading ways?

2) if yes. parameters need set in hive-site.xml file?.

3) can hive handle concurrent jdbc statements?

this exception i'm getting

java.sql.sqlexception: org.apache.thrift.transport.ttransportexception: read negative frame size (-2147418110)!     @ org.apache.hive.jdbc.hivestatement.execute(hivestatement.java:262)     @ org.ecl.dao.hadoopdboperationsdao.insertnonutivaluematchtrades_intotpcp(hadoopdboperationsdao.java:133)     @ org.ecl.service.dtccservice_multithreading_runnable.callhadoopserviceforloaddata(dtccservice_multithreading_runnable.java:133)     @ org.ecl.dtcchiveclient.main(dtcchiveclient.java:26) caused by: org.apache.thrift.transport.ttransportexception: read negative frame size (-2147418110)!     @ org.apache.thrift.transport.tsasltransport.readframe(tsasltransport.java:426)     @ org.apache.thrift.transport.tsasltransport.read(tsasltransport.java:405)     @ org.apache.thrift.transport.tsaslclienttransport.read(tsaslclienttransport.java:37)     @ org.apache.thrift.transport.ttransport.readall(ttransport.java:84)     @ org.apache.thrift.protocol.tbinaryprotocol.readall(tbinaryprotocol.java:378)     @ org.apache.thrift.protocol.tbinaryprotocol.readi32(tbinaryprotocol.java:297)     @ org.apache.thrift.protocol.tbinaryprotocol.readmessagebegin(tbinaryprotocol.java:204)     @ org.apache.thrift.tserviceclient.receivebase(tserviceclient.java:69)     @ org.apache.hive.service.cli.thrift.tcliservice$client.recv_executestatement(tcliservice.java:225)     @ org.apache.hive.service.cli.thrift.tcliservice$client.executestatement(tcliservice.java:212)     @ org.apache.hive.jdbc.hivestatement.execute(hivestatement.java:253)     ... 3 more java.sql.sqlexception: org.apache.thrift.transport.ttransportexception: read negative frame size (-2147418110)!     @ org.apache.hive.jdbc.hivestatement.execute(hivestatement.java:262)     @ org.ecl.dao.dtccpairinglogicdao.fieldbreak(dtccpairinglogicdao.java:60)     @ org.ecl.thread.fieldbreakthread.run(fieldbreakthread.java:41)     @ java.lang.thread.run(unknown source) caused by: org.apache.thrift.transport.ttransportexception: read negative frame size (-2147418110)!     @ org.apache.thrift.transport.tsasltransport.readframe(tsasltransport.java:426)     @ org.apache.thrift.transport.tsasltransport.read(tsasltransport.java:405)     @ org.apache.thrift.transport.tsaslclienttransport.read(tsaslclienttransport.java:37)     @ org.apache.thrift.transport.ttransport.readall(ttransport.java:84)     @ org.apache.thrift.protocol.tbinaryprotocol.readall(tbinaryprotocol.java:378)     @ org.apache.thrift.protocol.tbinaryprotocol.readi32(tbinaryprotocol.java:297)     @ org.apache.thrift.protocol.tbinaryprotocol.readmessagebegin(tbinaryprotocol.java:204)     @ org.apache.thrift.tserviceclient.receivebase(tserviceclient.java:69)     @ org.apache.hive.service.cli.thrift.tcliservice$client.recv_executestatement(tcliservice.java:225)     @ org.apache.hive.service.cli.thrift.tcliservice$client.executestatement(tcliservice.java:212)     @ org.apache.hive.jdbc.hivestatement.execute(hivestatement.java:253)     ... 3 more java.sql.sqlexception: org.apache.thrift.tapplicationexception: executestatement failed: out of sequence response     @ org.apache.hive.jdbc.hivestatement.execute(hivestatement.java:262)     @ org.ecl.dao.hadoopdboperationsdao.recordswithmatchedutivalue(hadoopdboperationsdao.java:149)     @ org.ecl.thread.utivaluematchedthread.run(utivaluematchedthread.java:40)     @ java.lang.thread.run(unknown source) caused by: org.apache.thrift.tapplicationexception: executestatement failed: out of sequence response     @ org.apache.thrift.tserviceclient.receivebase(tserviceclient.java:76)     @ org.apache.hive.service.cli.thrift.tcliservice$client.recv_executestatement(tcliservice.java:225)     @ org.apache.hive.service.cli.thrift.tcliservice$client.executestatement(tcliservice.java:212)     @ org.apache.hive.jdbc.hivestatement.execute(hivestatement.java:253)     ... 3 more 0 trade records breaks identified in trade records matching on uti java.sql.sqlexception: org.apache.thrift.transport.ttransportexception: cannot read. remote side has closed. tried read 3072 bytes, got 175 bytes. (this indicative of internal error on server side. please check server logs.)     @ org.apache.hive.jdbc.hivestatement.execute(hivestatement.java:262)     @ org.apache.hive.jdbc.hivestatement.executequery(hivestatement.java:392)     @ org.ecl.dao.dtccoperationdao.countrows(dtccoperationdao.java:45)     @ org.ecl.thread.fieldbreakthread.run(fieldbreakthread.java:42)     @ java.lang.thread.run(unknown source) caused by: org.apache.thrift.transport.ttransportexception: cannot read. remote side has closed. tried read 3072 bytes, got 175 bytes. (this indicative of internal error on server side. please check server logs.)     @ org.apache.thrift.transport.ttransport.readall(ttransport.java:86)     @ org.apache.thrift.protocol.tbinaryprotocol.readstringbody(tbinaryprotocol.java:354)     @ org.apache.thrift.protocol.tbinaryprotocol.readmessagebegin(tbinaryprotocol.java:215)     @ org.apache.thrift.tserviceclient.receivebase(tserviceclient.java:69)     @ org.apache.hive.service.cli.thrift.tcliservice$client.recv_executestatement(tcliservice.java:225)     @ org.apache.hive.service.cli.thrift.tcliservice$client.executestatement(tcliservice.java:212)     @ org.apache.hive.jdbc.hivestatement.execute(hivestatement.java:253)     ... 4 more }

so me resolve this.


Comments

Post a Comment

Popular posts from this blog

qt - Using float or double for own QML classes -

-
i've created components , helper methods using c++ , qt used in qml gui. of methods calculate gui elements (size, transformations, scene graph items creation). target arm 32 bit using opengl / eglfs render output. opengl using float , qml using real (defined double ). what should use in c++ do not lose precision? if use qreal ( double ) shouldn't lose precision. if git grep float in qtbase.git , can find out bit more: dist/changes-5.2.0-**************************************************************************** dist/changes-5.2.0-* important behavior changes * dist/changes-5.2.0-**************************************************************************** dist/changes-5.2.0- dist/changes-5.2.0- - qt compiled qreal typedef'ed double on dist/changes-5.2.0: platforms. qreal float on arm chipsets before. guarantees more dist/changes-5.2.0- consistent behavior between platforms qt supports, binary dist/chan...
Read more

Create Outlook appointment via C# .Net -

-
i need create outlook appointment using microsoft.office.interop.outlook , while can work locally on own workstation , works if run web application via server's browser, not work when connect server externally. hence, think might permission issue. i changed application pool identity " localsystem " don't access denied error. unfortunately, doesn't work. the app behaves if appointment created, appointment doesn't appear in outlook. here include file on top of aspx.cs page using outlook = microsoft.office.interop.outlook; here code using pop appointment. outlook.application apptapp = new outlook.application(); outlook.appointmentitem appt = apptapp.createitem(outlook.olitemtype.olappointmentitem) outlook.appointmentitem; appt.subject = txtfirstname.text + " " + txtlastname.text; appt.body = txtcomment.text; appt.alldayevent = false; appt.start = datetime.parse(txtreminderdate.text + " 8:00 am"); appt.end = datetime.p...
Read more

ios - Swift Array Resetting Itself -

-
i have 2 arrays: pointhistory , datehistory . have nstimer , , when timer runs out, action called. in action appending int pointhistory , , current date string datehistory . here timer action: func timeraction() { let steppervalue = int(stepper.value) employee.numberofpoints = steppervalue networking.saveemployee(employee, completion: { (error) -> void in if error != nil { let alert = uialertcontroller(title: "error", message: "there error updating \(self.employee.name)'s points", preferredstyle: .alert) let alertaction = uialertaction(title: "ok", style: .cancel, handler: nil) alert.addaction(alertaction) self.presentviewcontroller(alert, animated: true, completion: nil) } }) var dateformatter = nsdateformatter() dateformatter.dateformat = "mm-dd-yyyy" dateformatter.timezone = nstimezone.localtimezone() var date = nsdate() var ...
Read more