evaluation


Predictionio evaluation fails with Text Classification template


I am trying to predict a text field based on other text fields on predictionio. I used this guide for reference. I created a new app using
pio app new MyTextApp
and followed the guide upto evaluation using datasource provided in template. It was all okay upto evaluation. On evaluating data source I am getting error as pasted below.
[INFO] [CoreWorkflow$] runEvaluation started
[WARN] [Utils] Your hostname, my-ThinkCentre-Edge72 resolves to a loopback address: 127.0.0.1; using 192.168.65.27 instead (on interface eth0)
[WARN] [Utils] Set SPARK_LOCAL_IP if you need to bind to another address
[INFO] [Remoting] Starting remoting
[INFO] [Remoting] Remoting started; listening on addresses :[akka.tcp://sparkDriver#192.168.65.27:59649]
[INFO] [CoreWorkflow$] Starting evaluation instance ID: AU29p8j3Fkwdnkfum_ke
[INFO] [Engine$] DataSource: org.template.textclassification.DataSource#faea4da
[INFO] [Engine$] Preparator: org.template.textclassification.Preparator#69f2cb04
[INFO] [Engine$] AlgorithmList: List(org.template.textclassification.NBAlgorithm#45292ec1)
[INFO] [Engine$] Serving: org.template.textclassification.Serving#1ad9b8d3
Exception in thread "main" java.lang.UnsupportedOperationException: empty.maxBy
at scala.collection.TraversableOnce$class.maxBy(TraversableOnce.scala:223)
at scala.collection.AbstractTraversable.maxBy(Traversable.scala:105)
at org.template.textclassification.PreparedData.<init>(Preparator.scala:152)
at org.template.textclassification.Preparator.prepare(Preparator.scala:38)
at org.template.textclassification.Preparator.prepare(Preparator.scala:34)
Do I have to edit any config files to make this work? I have successfully ran tests on movielens data.
So this particular error message occurs when your data isn't getting read properly through the DataSource class. If you're using a different text data set, then make sure that you are correctly reflecting any changes to the eventNames, entityType, and respective property field names in the readEventData method.
The maxBy method is used to pull the class with the highest number of observations. If the category to label Map is empty, it means that there are no classes being recorded, which essentially tells you have no data being fed in.
For example, I just did a spam detector using this engine. My e-mail data is of the form:
{"entityType": "content", "eventTime": "2015-06-04T00:22:39.064+0000", "entityId": 1, "event": "e-mail", "properties": {"label": "spam", "text": "content"}}
To use the engine for this data I made the following changes in the DataSource class:
entityType = Some("source"), // specify data entity type
eventNames = Some(List("documents")) // specify data event name
changes to
entityType = Some("content"), // specify data entity type
eventNames = Some(List("e-mail")) // specify data event name
and
)(sc).map(e => Observation(
e.properties.get[Double]("label"),
e.properties.get[String]("text"),
e.properties.get[String]("category")
)).cache
changes to:
)(sc).map(e => {
val label = e.properties.get[String]("label")
Observation(
if (label == "spam") 1.0 else 0.0,
e.properties.get[String]("text"),
label
)
}).cache
After this, I'm able to go through building, training, and deployment, as well as an evaluation.

Related Links

SyntaxNet evaluation metrics
How am I misunderstanding call-by-need evaluation?
Math notation understanding
What criterias do you look for in a WAF (web application firewall)?
Predictionio evaluation fails with Text Classification template
Evaluation of APL direct functions
Modelica Evaluation Order
how compilers evaluate mathematical expressions? [closed]
What does “top-level evaluation” mean?
Boolean expression evaluation with a given context
Is relying on short-circuit evaluation good design? [closed]
Objectively evaluating new GUI toolkits
what are evaluation techniques for HCI projects (without involving users, or psychologist)?
Evaluating developers [closed]
Options to Common Criteria
What are some useful criteria for deciding on which software package to go for?

Categories

HOME
oop
ssms
stata
windows-server-2008
python-requests
squeak
overrides
opacity
collision-detection
virtualhost
sitemap
revit-api
software-collections
segmentation-fault
wowza
fine-uploader
geography
audio-streaming
summernote
custom-component
priority-queue
tftp
emr
apm
arp
contextmenustrip
wit-ai
sampling
aweber
tapply
reqif
ocamlbuild
automapper-4
nao-robot
ng-class
serialversionuid
cancan
openpgp.js
typewriter
androiddesignsupport
jtextarea
init
airwatch
android-xmlpullparser
nimbus
supervised-learning
camellia
jmap
nd4j
innertext
datamodel
mojolicious
log4cxx
litedb
grails-3.0.9
connection-timeout
wingdb
chunks
touch-typing
teamstudio-unplugged
pysvn
aiff
dotcmis
myrrix
raddatepicker
aspnet-compiler
kendo-window
objectdatasource
angularjs-select2
biztalk-deployment
dnsbl
invalidoperationexception
event-propagation
nsscrollview
rjs
digg
hwnd
xhtmlrenderer
conditional-operator
mysql-logic
large-data-volumes
morfik
html-help-workshop
cstring
managed-code
httpbrowsercapabilities

Resources

Mobile Apps Dev
Database Users
javascript
java
csharp
php
android
MS Developer
developer works
python
ios
c
html
jquery
RDBMS discuss
Cloud Virtualization
Database Dev&Adm
javascript
java
csharp
php
python
android
jquery
ruby
ios
html
Mobile App
Mobile App
Mobile App