Open a terminal and paste this commands. Latest version of Cassandra until this day is 3.9. Check the datastax page to see which is the latest version or you can go to this link and see the latest release_X.Y version. In this case i can see a release_3.9.conf file, so latest version is 3.9. echo […]
Hi,i just have to play and learn how to use this algorithm provided by spark-ml to do some feature extractions from some text using Google`s Word2Vec algorithm, i mean, why not to use my actual cv? Before that, probably you will have to convert the pdf file to text file. Actually i am working with […]
You basically need to make a Pipeline and build a ParamGrid with different algorithms as stages. Here is an simple example: val dt = new DecisionTreeClassifier() .setLabelCol(“label”) .setFeaturesCol(“features”) val lr = new LogisticRegression() .setLabelCol(“label”) .setFeaturesCol(“features”) val pipeline = new Pipeline() val paramGrid = new ParamGridBuilder() .addGrid(pipeline.stages, Array(Array[PipelineStage](dt), Array[PipelineStage](lr))) val cv = new CrossValidator() .setEstimator(pipeline) .setEstimatorParamMaps(paramGrid) […]
For some time I did not update the blog, so I want to add a small repository in github on how to create a parquet file using the spark-csv library of databricks. Super easy to use, so simple that the code is self-explanatory. https://github.com/alonsoir/parquet-utils
Hi everyone, as usual, I put here a recipe to create fast a table in Hive using the data from a data frame. I am using a spark-shell connected to a development cluster, cloudera version is cdh5.5.2, so, according to this official cloudera site, hive version is 1.1.0. It should be important when I have […]
Twitter Sentiment Analysis ========================== * retrieve tweets using Spark Streaming * language detection * sentiment analysis (StanfordNLP) * index tweets in Elasticsearch * live dashboard using Kibana First at all, thank you to Vincent Spiewak, original author of this project. I have modified a bit the code in order to run it in local mode […]
Hi, recently i was receiving classes from formacionhadoop.com, Master online big data expert, 150 hours, which means that i received notions about hadoop, spark and nosql databases. A good course, i recommend it to everyone to learn the basis of big data technology. I already have taken classes from Andrew Ng with its Machine Learning coursera course. […]
