AN EVENT-DRIVEN, SCALABLE, NEAR-REAL-TIME, HIGH-AVAILABILITY, PREDICTIVE IMAGE ARCHITECTURE.

Lately I've been looking at little things to see which is the best way for an event-oriented architecture (with Confluent platform, for example) to get predictions on images, besides fighting with Kubernetes and the confluence platform, you know, a way to have packaged and super easy a Kafka/Zookeeper Cluster, while encouraging you to use that…

Advertisement

Playing with Word2Vec, my cv, spark and scala

Hi,i just have to play and learn how to use this algorithm provided by spark-ml to do some feature extractions from some text using Google`s Word2Vec algorithm, i mean, why not to use my actual cv? Before that, probably you will have to convert the pdf file to text file. Actually i am working with…

About how to parallelize multiple Machine Learning Algorithm using a pipeline with spark.

You basically need to make a Pipeline and build a ParamGrid with different algorithms as stages.  Here is an simple example: val dt = new DecisionTreeClassifier() .setLabelCol("label") .setFeaturesCol("features") val lr = new LogisticRegression() .setLabelCol("label") .setFeaturesCol("features") val pipeline = new Pipeline() val paramGrid = new ParamGridBuilder() .addGrid(pipeline.stages, Array(Array[PipelineStage](dt), Array[PipelineStage](lr))) val cv = new CrossValidator() .setEstimator(pipeline) .setEstimatorParamMaps(paramGrid)…