evaluation


Predictionio evaluation fails with Text Classification template


I am trying to predict a text field based on other text fields on predictionio. I used this guide for reference. I created a new app using
pio app new MyTextApp
and followed the guide upto evaluation using datasource provided in template. It was all okay upto evaluation. On evaluating data source I am getting error as pasted below.
[INFO] [CoreWorkflow$] runEvaluation started
[WARN] [Utils] Your hostname, my-ThinkCentre-Edge72 resolves to a loopback address: 127.0.0.1; using 192.168.65.27 instead (on interface eth0)
[WARN] [Utils] Set SPARK_LOCAL_IP if you need to bind to another address
[INFO] [Remoting] Starting remoting
[INFO] [Remoting] Remoting started; listening on addresses :[akka.tcp://sparkDriver#192.168.65.27:59649]
[INFO] [CoreWorkflow$] Starting evaluation instance ID: AU29p8j3Fkwdnkfum_ke
[INFO] [Engine$] DataSource: org.template.textclassification.DataSource#faea4da
[INFO] [Engine$] Preparator: org.template.textclassification.Preparator#69f2cb04
[INFO] [Engine$] AlgorithmList: List(org.template.textclassification.NBAlgorithm#45292ec1)
[INFO] [Engine$] Serving: org.template.textclassification.Serving#1ad9b8d3
Exception in thread "main" java.lang.UnsupportedOperationException: empty.maxBy
at scala.collection.TraversableOnce$class.maxBy(TraversableOnce.scala:223)
at scala.collection.AbstractTraversable.maxBy(Traversable.scala:105)
at org.template.textclassification.PreparedData.<init>(Preparator.scala:152)
at org.template.textclassification.Preparator.prepare(Preparator.scala:38)
at org.template.textclassification.Preparator.prepare(Preparator.scala:34)
Do I have to edit any config files to make this work? I have successfully ran tests on movielens data.
So this particular error message occurs when your data isn't getting read properly through the DataSource class. If you're using a different text data set, then make sure that you are correctly reflecting any changes to the eventNames, entityType, and respective property field names in the readEventData method.
The maxBy method is used to pull the class with the highest number of observations. If the category to label Map is empty, it means that there are no classes being recorded, which essentially tells you have no data being fed in.
For example, I just did a spam detector using this engine. My e-mail data is of the form:
{"entityType": "content", "eventTime": "2015-06-04T00:22:39.064+0000", "entityId": 1, "event": "e-mail", "properties": {"label": "spam", "text": "content"}}
To use the engine for this data I made the following changes in the DataSource class:
entityType = Some("source"), // specify data entity type
eventNames = Some(List("documents")) // specify data event name
changes to
entityType = Some("content"), // specify data entity type
eventNames = Some(List("e-mail")) // specify data event name
and
)(sc).map(e => Observation(
e.properties.get[Double]("label"),
e.properties.get[String]("text"),
e.properties.get[String]("category")
)).cache
changes to:
)(sc).map(e => {
val label = e.properties.get[String]("label")
Observation(
if (label == "spam") 1.0 else 0.0,
e.properties.get[String]("text"),
label
)
}).cache
After this, I'm able to go through building, training, and deployment, as well as an evaluation.

Related Links

SyntaxNet evaluation metrics
How am I misunderstanding call-by-need evaluation?
Math notation understanding
What criterias do you look for in a WAF (web application firewall)?
Predictionio evaluation fails with Text Classification template
Evaluation of APL direct functions
Modelica Evaluation Order
how compilers evaluate mathematical expressions? [closed]
What does “top-level evaluation” mean?
Boolean expression evaluation with a given context
Is relying on short-circuit evaluation good design? [closed]
Objectively evaluating new GUI toolkits
what are evaluation techniques for HCI projects (without involving users, or psychologist)?
Evaluating developers [closed]
Options to Common Criteria
What are some useful criteria for deciding on which software package to go for?

Categories

HOME
joomla
jenkins-pipeline
nagios
arcgis
arguments
msbi
perl6
checkpoint
install
jcodemodel
kairosdb
velocity
logstash-configuration
responsive
facebook-marketing-api
indri
fibonacci
javaagents
connection-pooling
bootstrap-accordion
struts
apdu
datastax-enterprise
knapsack-problem
tftp
multiple-inheritance
log4net-configuration
contextmenustrip
ilog
ponylang
respect-validation
rtsp
area
mybatis-generator
ng-lightning
keychain
definition
python-venv
taleo
g77
dcom
extjs3
e4
audio-fingerprinting
printer-control-language
node.js-client
breadcrumbs
hoare-logic
system-integration
usps
jmh
firefox-os
dday
google-app-engine-php
computability
producer-consumer
javapackager
gulp-cdnizer
lytro
jain-sip
openhab
polarion
firewire
fileinputstream
connection-timeout
jsonschema2pojo
livescript
disabled-control
amzi-prolog
ant-contrib
mdp
xml-validation
friendly-id
infomaker
spawn
reference-manual
database-restore
moonscript
custom-cursor
sup
optimistic-concurrency
eeprom
unordered-set
jqzoom
openwrap
execjs
youtube.net-api
hs-err
mass-emails
simultaneous
xslt-grouping
for-xml
manuals
html-help-workshop

Resources

Mobile Apps Dev
Database Users
javascript
java
csharp
php
android
MS Developer
developer works
python
ios
c
html
jquery
RDBMS discuss
Cloud Virtualization
Database Dev&Adm
javascript
java
csharp
php
python
android
jquery
ruby
ios
html
Mobile App
Mobile App
Mobile App