跳至主要内容

博文

目前显示的是标签为“mllib”的博文

Enabling Native Acceleration for MLlib

The undefined symbol issue Got onto the ship of machine learning. And soon I hit the wall of `undefined symbol issue’ on lab cluster. Hi, it just the simple `MinMaxScaler’ example code published on spark MLlib(Machine Learning) guide!! Everything goes well until the last line `scaledData.show()`, boom. Spark-shell died with the following message on the console: /usr/java/jdk1.7.0_67-cloudera/bin/java: symbol lookup error: /tmp/jniloader82069440205403545netlib-native_system-linux-x86_64.so: undefined symbol: cblas_daxpy NO log. NO history server record. Nothing could be used for debug as first glance. Solution (Wrap up) My solution is based on CDH 5.11.0 (parcel) plus cloudera GPLExtra (parcel) plus Intel MKL library (parcel). The steps to enable MKL native acceleration for cloudera spark should be as simple as: Install `netlib-java` by integrate GPLExtra parcel as described in the Enable Native Accerleration For MLlib Install MKL library parcel by follow the Download Intel Ma...