Ubuntu/UbuntuKylin平台下源碼編譯Ambari 2.4(含較多問題解決方案)


1、Ubuntu環境配置 1.1 網絡配置 kylin@Ubuntu:~$ sudo vim /etc/network/interfaces \# interfaces(5) file used by ifup(8) and ifdown(8) auto ens32 iface ens32 inet static address 192.168.1.166 gateway 192.168.1.1 netmask 255.255.255.0 network 192.168.1.0 broadcast 192.168.1.255 重啟系統 kylin@Ubuntu:~$ sudo /etc/init.d/networking restart [ ok ] Restarting networking (via systemctl): networking.service. 1.2 Ubuntu安裝SSH sudoapt−getinstallssh sudo apt-get install rsync kylin@Ubuntu:~$ dpkg -l | grep ssh ii  libssh-4:amd64                              0.7.3-1                                     amd64        tiny C SSH library (OpenSSL flavor) ii  libssh-gcrypt-4:amd64                       0.7.3-1                                     amd64        tiny C SSH library (gcrypt flavor) ii  openssh-client                              1:7.3p1-1                                   amd64        secure shell (SSH) client, for secure access to remote machines ii  openssh-server                              1:7.3p1-1                                   amd64        secure shell (SSH) server, for secure access from remote machines ii  openssh-sftp-server                         1:7.3p1-1                                   amd64        secure shell (SSH) sftp server module, for SFTP access from remote machines ii  ssh                                         1:7.3p1-1                                   all          secure shell client and server (metapackage) ii  ssh-import-id                               5.6-0ubuntu1                                all          securely retrieve an SSH public key and install it locally 1.3 如何讓ubuntukylin開機進入純命令模式 找了搜索了很多方法都是針對ubuntu,ubuntukylin沒有用!! kylin@Ubuntu:~sudoservicelightdmstartkylin@Ubuntu:  sudo lightdm start Failed to use bus name org.freedesktop.DisplayManager, do you have appropriate permissions? 1.4 JDK kylin@Ubuntu:~$ dpkg -l | grep jdk kylin@Ubuntu:~$ dpkg -l | grep java ii  gir1.2-javascriptcoregtk-4.0:amd64          2.14.0-1                                    amd64        JavaScript engine library from WebKitGTK+ - GObject introspection data ii  libjavascriptcoregtk-1.0-0:amd64            2.4.11-3                                    amd64        JavaScript engine library from WebKitGTK+ ii  libjavascriptcoregtk-4.0-18:amd64           2.14.0-1                                    amd64        JavaScript engine library from WebKitGTK+ kylin@Ubuntu:~$  kylin@Ubuntu:~$ java -version 程序 'java' 已包含在下列軟件包中: * default-jre * gcj-5-jre-headless * openjdk-8-jre-headless * gcj-4.8-jre-headless * gcj-4.9-jre-headless * gcj-6-jre-headless * openjdk-9-jre-headless 請嘗試:sudo apt install <選定的軟件包> kylin@Ubuntu:~$ sudo vi .profile #set for java export JAVA_HOME=/home/kylin/jdk1.7.0_67 export CLASSPATH=.:$CLASSPATH:$JAVA_HOME/lib export _JAVA_OPTIONS="-Xmx2048m -XX:MaxPermSize=512m -Djava.awt.headless=true" export PATH=$PATH:$JAVA_HOME/bin kylin@Ubuntu:~$ source .profile kylin@Ubuntu:~$ java -version java version "1.7.0_67" Java(TM) SE Runtime Environment (build 1.7.0_67-b01) Java HotSpot(TM) 64-Bit Server VM (build 24.65-b04, mixed mode) 1.5 Maven kylin@Ubuntu:~$ vi .profile  #set for maven export MAVEN_HOME=/home/kylin/apache-maven-3.0.5 export PATH=$PATH:$MAVEN_HOME/bin kylin@Ubuntu:~$ source .profile  maven有2個位置可以選擇配置M2_HOME/conf/setting.xml和~/.m2/setting.xml 前者是全局性的,后者是針對用戶的。 查詢資料是說用戶配置優先級高於全局配置 kylin@Ubuntu:~/apache-maven-3.0.5/conf$ cp settings.xml ~/.m2/ 1.6 Node.js kylin@Ubuntu:~$ sudo tar -zxvf node-v0.10.44-linux-x64.tar.gz -C /opt kylin@Ubuntu:~$ vi .profile  #set for nodejs export NODE_HOME=/home/kylin/node-v0.10.44-linux-x64 export PATH=$PATH:$NODE_HOME/bin kylin@Ubuntu:~$ source .profile  配置淘寶源 kylin@Ubuntu:~$ npm config set registry https://registry.npm.taobao.org   kylin@Ubuntu:~$ npm info underscor  安裝brunch kylin@Ubuntu:~$ npm install -g brunch@1.7.20 npm WARN deprecated minimatch@1.0.0: Please update to minimatch 3.0.2 or higher to avoid a RegExp DoS issue npm WARN optional dep failed, continuing fsevents@0.3.8 npm WARN deprecated minimatch@0.2.14: Please update to minimatch 3.0.2 or higher to avoid a RegExp DoS issue npm WARN deprecated graceful-fs@2.0.3: graceful-fs v3.0.0 and before will fail on node releases >= v7.0. Please update to graceful-fs@^4.0.0 as soon as possible. Use 'npm ls graceful-fs' to find it in the tree. /home/kylin/node-v0.10.44-linux-x64/bin/brunch -> /home/kylin/node-v0.10.44-linux-x64/lib/node_modules/brunch/bin/brunch > brunch@1.7.20 postinstall /home/kylin/node-v0.10.44-linux-x64/lib/node_modules/brunch > node setup.js postinstall brunch@1.7.20 /home/kylin/node-v0.10.44-linux-x64/lib/node_modules/brunch ├── debug@0.7.4 ├── async-waterfall@0.1.5 ├── commonjs-require-definition@0.1.2 ├── async-each@0.1.6 ├── commander@2.0.0 ├── ncp@0.4.2 ├── mkdirp@0.3.5 ├── init-skeleton@0.2.4 (rimraf@2.2.8) ├── read-components@0.6.1 (component-builder@0.10.1) ├── source-map@0.1.43 (amdefine@1.0.1) ├── coffee-script@1.8.0 ├── anymatch@1.0.0 (minimatch@1.0.0) ├── chokidar@0.12.6 (readdirp@1.3.0) ├── anysort@1.0.1 (anymatch@1.3.0) ├── pushserve@0.1.6 (connect-slashes@0.0.11, express@3.3.8) └── loggy@0.2.2 (growl@1.8.1, ansi-color@0.2.1, date-utils@1.2.21) 1.7 安裝rpmbuild ambari編譯時依賴rpmbuild打包成rpm安裝包,因而需要rpm套件,很幸運,ubuntu可以安裝這一套件: kylin@Ubuntu:~$ sudo apt-get install -y rpm yum 1.8 Python Ubuntu默認是Python 2.7,直接設置setuptools-0.6c11-py2.7.egg kylin@Ubuntu:~$ sudo sh setuptools-0.6c11-py2.7.egg Processing setuptools-0.6c11-py2.7.egg Copying setuptools-0.6c11-py2.7.egg to /usr/local/lib/python2.7/dist-packages Adding setuptools 0.6c11 to easy-install.pth file Installing easy_install script to /usr/local/bin Installing easy_install-2.7 script to /usr/local/bin Installed /usr/local/lib/python2.7/dist-packages/setuptools-0.6c11-py2.7.egg Processing dependencies for setuptools==0.6c11 Finished processing dependencies for setuptools==0.6c11 1.9 git kylin@Ubuntu:~$ sudo apt-get install -y git   2、源碼編譯Ambari 2.1 下載源碼 apache-ambari-2.4.2-src.tar.gz 解壓縮 kylin@Ubuntu:~$ tar -zxvf apache-ambari-2.4.2-src.tar.gz 2.2 安裝node-gyp 進入源碼根目錄安裝node-gyp kylin@Ubuntu:~$ cd apache-ambari-2.4.2-src/ kylin@Ubuntu:~/apache-ambari-2.4.2-src$ ls ambari-admin  ambari-client  ambari-funtest    ambari-metrics  ambari-server  ambari-views  contrib      docs  LICENSE.txt  NOTICE.txt  pom.xml ambari-agent  ambari-common  ambari-logsearch  ambari-project  ambari-shell   ambari-web    dev-support  KEYS  logo.png     pass.txt    version kylin@Ubuntu:~/apache-ambari-2.4.2-src$ npm install node-gyp 2.3 准備編譯 kylin@Ubuntu:~/apache-ambari-2.4.2-src$ mvn versions:set -DnewVersion=2.4.2.0.0 kylin@Ubuntu:~/apache-ambari-2.4.2-src$ pushd ambari-metrics ~/apache-ambari-2.4.2-src/ambari-metrics ~/apache-ambari-2.4.2-src kylin@Ubuntu:~/apache-ambari-2.4.2-src/ambari-metrics$ cd .. kylin@Ubuntu:~/apache-ambari-2.4.2-src$ popd ~/apache-ambari-2.4.2-src 2.4 開始編譯 kylin@Ubuntu:~/apache-ambari-2.4.2-src$ mvn -B clean install package jdeb:jdeb -DskipTests -Dpython.ver="python >= 2.6" -Preplaceurl   3、遇到問題: 問題1、Too many files with unapproved license [ERROR] Failed to execute goal org.apache.rat:apache-rat-plugin:0.11:check (default) on project ambari: Too many files with unapproved license: 780 See RAT report in: /home/kylin/apache-ambari-2.4.2-src/target/rat.txt -> [Help 1] 解決辦法:加上-Drat.skip=true kylin@Ubuntu:~/apache-ambari-2.4.2-src$ mvn -B install package jdeb:jdeb -DskipTests -Dpython.ver="python >= 2.6" -Drat.skip=true -Preplaceurl 問題2、#include 問題3、UnknownHostException Unknown host 問題3.1 Unknown host grafanarel.s3.amazonaws.com [ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (default) on project ambari-metrics-grafana: An Ant BuildException has occured: Java.NET.UnknownHostException: grafanarel.s3.amazonaws.com [ERROR] around Ant part …… @ 5:217 in /home/kylin/apache-ambari-2.4.2-src/ambari-metrics/ambari-metrics-grafana/target/antrun/build-Download Ambari Grafana.xml: Unknown host grafanarel.s3.amazonaws.com 問題3.2 Unknown host private-repo-1.hortonworks.com [ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (default) on project ambari-metrics-timelineservice: An Ant BuildException has occured: java.Net.UnknownHostException: private-repo-1.hortonworks.com [ERROR] around Ant part …… @ 5:265 in /home/kylin/apache-ambari-2.4.2-src/ambari-metrics/ambari-metrics-timelineservice/target/antrun/build-Download Hbase.xml: Unknown host private-repo-1.hortonworks.com 問題3.3 Unknown host www.w3.org [ERROR] Failed to execute goal org.codehaus.mojo:xml-maven-plugin:1.0:transform (default) on project ambari-server: Failed to transform input file /home/kylin/apache-ambari-2.4.2-src/ambari-server/target/findbugs/findbugsXml.html: java.net.UnknownHostException: www.w3.org: Unknown host www.w3.org -> [Help 1] 解決辦法:重新執行 kylin@Ubuntu:~/apache-ambari-2.4.2-src$ mvn -B install package jdeb:jdeb -DnewVersion=2.4.2.0.0 -DskipTests -Dpython.ver="python >= 2.6" -Drat.skip=true 問題4、findbugsXml.html [ERROR] Failed to execute goal org.codehaus.mojo:xml-maven-plugin:1.0:transform (default) on project ambari-server: Failed to transform input file /home/kylin/apache-ambari-2.4.2-src/ambari-server/target/findbugs/findbugsXml.html: The entity name must immediately follow the ‘&’ in the entity reference. -> [Help 1] 解決辦法1: 重新執行 kylin@Ubuntu:~/apache-ambari-2.4.2-src$ mvn -B install package jdeb:jdeb -DnewVersion=2.4.2.0.0 -DskipTests -Dpython.ver="python >= 2.6" -Drat.skip=true 解決辦法2: 如果網絡情況不好,這一步就很難過去。那只好修改pom.xml文件,將findbugs內容注釋掉: kylin@Ubuntu:~/apache-ambari-2.4.2-src/ambari-server$ vim pom.xml <plugin> <groupId>org.codehaus.mojo</groupId> <artifactId>findbugs-maven-plugin</artifactId> <version>3.0.3</version> <configuration> <failOnError>false</failOnError> <threshold>Low</threshold> <findbugsXmlOutputDirectory>${project.basedir}/target/findbugs</findbugsXmlOutputDirectory> </configuration> <executions> <!--   <execution> <phase>verify</phase> <goals> <goal>check</goal> </goals> </execution> --> </executions> </plugin> <plugin> <groupId>org.codehaus.mojo</groupId> <artifactId>xml-maven-plugin</artifactId> <version>1.0</version> <executions> <!-- <execution> <phase>verify</phase> <goals> <goal>transform</goal> </goals> </execution> --> </executions> 問題5、 is not a valid ‘control’ directory 參考:https://issues.apache.org/jira/browse/AMBARI-12977 問題5.1 ambari-metrics-grafana Caused by: org.vafer.jdeb.PackagingException: “/home/kylin/apache-ambari-2.4.2-src/ambari-metrics/ambari-metrics-grafana/src/main/package/deb/control” is not a valid ‘control’ directory) kylin@Ubuntu:~/apache-ambari-2.4.2-src/ambari-metrics/ambari-metrics-grafana$ vim pom.xml 在最后添加一個添加,如下代碼 <plugin> <groupId>org.vafer</groupId> <artifactId>jdeb</artifactId> <version>1.0.1</version> <executions> <execution> <!--Stub execution on direct plugin call - workaround for ambari deb build process--> <id>stub-execution</id> <phase>none</phase> <goals> <goal>jdeb</goal> </goals> </execution> </executions> <configuration> <skip>true</skip> <attach>false</attach> <submodules>false</submodules> <controlDir>${project.basedir}/../src/main/package/deb/control</controlDir> </configuration> </plugin> </plugins> </build> 問題5.2 ambari-funtest [ERROR] Failed to create debian package /home/kylin/apache-ambari-2.4.2-src/ambari-funtest/target/ambari-funtest_2.4.2.0.0_all.deb org.vafer.jdeb.PackagingException: “/home/kylin/apache-ambari-2.4.2-src/ambari-funtest/src/main/package/deb/control” is not a valid ‘control’ directory) 解決辦法:在上面添加一個元素 kylin@Ubuntu:~/apache-ambari-2.4.2-src/ambari-funtest$ vi pom.xml <plugin> <groupId>org.vafer</groupId> <artifactId>jdeb</artifactId> <version>1.0.1</version> <executions> <execution> <!--Stub execution on direct plugin call - workaround for ambari deb build process--> <id>stub-execution</id> <phase>none</phase> <goals> <goal>jdeb</goal> </goals> </execution> </executions> <configuration> <skip>true</skip> <attach>false</attach> <submodules>false</submodules> <controlDir>${project.basedir}/../src/main/package/deb/control</controlDir> </configuration> </plugin> </plugins> <resources> <resource> 問題5.3 ambari-logsearch [ERROR] Failed to execute goal org.vafer:jdeb:1.0.1:jdeb (default-cli) on project ambari-logsearch: Failed to create debian package /home/kylin/apache-ambari-2.4.2-src/ambari-logsearch/target/ambari-logsearch_2.4.2.0.0_all.deb: “/home/kylin/apache-ambari-2.4.2-src/ambari-logsearch/src/main/package/deb/control” is not a valid ‘control’ directory) -> [Help 1] 解決辦法:在上面添加一個元素 kylin@Ubuntu:~/apache-ambari-2.4.2-src/ambari-logsearch$ vim pom.xml <plugin> <groupId>org.vafer</groupId> <artifactId>jdeb</artifactId> <version>1.0.1</version> <executions> <execution> <!--Stub execution on direct plugin call - workaround for ambari deb build process--> <id>stub-execution</id> <phase>none</phase> <goals> <goal>jdeb</goal>
</goals>
</execution> </executions> <configuration> <skip>true</skip> <attach>false</attach> <submodules>false</submodules> <controlDir>${project.basedir}/../src/main/package/deb/control</controlDir> </configuration> </plugin> </plugins> </build>   4、編譯完成 [INFO] ------------------------------------------------------------------------ [INFO] Reactor Summary: [INFO]  [INFO] Ambari Main ....................................... SUCCESS [1.620s] [INFO] Apache Ambari Project POM ......................... SUCCESS [0.078s] [INFO] Ambari Web ........................................ SUCCESS [59.983s] [INFO] Ambari Views ...................................... SUCCESS [0.906s] [INFO] Ambari Admin View ................................. SUCCESS [19.604s] [INFO] ambari-metrics .................................... SUCCESS [0.434s] [INFO] Ambari Metrics Common ............................. SUCCESS [0.421s] [INFO] Ambari Metrics Hadoop Sink ........................ SUCCESS [2.242s] [INFO] Ambari Metrics Flume Sink ......................... SUCCESS [1.049s] [INFO] Ambari Metrics Kafka Sink ......................... SUCCESS [0.697s] [INFO] Ambari Metrics Storm Sink ......................... SUCCESS [2.175s] [INFO] Ambari Metrics Storm Sink (Legacy) ................ SUCCESS [1.617s] [INFO] Ambari Metrics Collector .......................... SUCCESS [13.404s] [INFO] Ambari Metrics Monitor ............................ SUCCESS [1.155s] [INFO] Ambari Metrics Grafana ............................ SUCCESS [15.134s] [INFO] Ambari Metrics Assembly ........................... SUCCESS [1:38.729s] [INFO] Ambari Server ..................................... SUCCESS [2:03.242s] [INFO] Ambari Functional Tests ........................... SUCCESS [0.445s] [INFO] Ambari Agent ...................................... SUCCESS [19.449s] [INFO] Ambari Client ..................................... SUCCESS [0.021s] [INFO] Ambari Python Client .............................. SUCCESS [0.741s] [INFO] Ambari Groovy Client .............................. SUCCESS [3.822s] [INFO] Ambari Shell ...................................... SUCCESS [0.056s] [INFO] Ambari Python Shell ............................... SUCCESS [0.744s] [INFO] Ambari Groovy Shell ............................... SUCCESS [1.118s] [INFO] ambari-logsearch .................................. SUCCESS [0.014s] [INFO] Ambari Logsearch Appender ......................... SUCCESS [0.287s] [INFO] Ambari Logsearch Solr Client ...................... SUCCESS [1.501s] [INFO] Ambari Logsearch Portal ........................... SUCCESS [3:58.085s] [INFO] Ambari Logsearch Log Feeder ....................... SUCCESS [7:39.905s] [INFO] Ambari Logsearch Assembly ......................... SUCCESS [0.155s] [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 17:49.903s [INFO] Finished at: Wed Mar 01 15:43:16 CST 2017 [INFO] Final Memory: 231M/1188M [INFO] ------------------------------------------------------------------------   備注: Ambari Server will create following packages RPM will be created under AMBARI_DIR/ambari-server/target/rpm/ambari-server/RPMS/noarch. DEB will be created under AMBARI_DIR/ambari-server/target/ Ambari Agent will create following packages RPM will be created under AMBARI_DIR/ambari-agent/target/rpm/ambari-agent/RPMS/x86_64. DEB will be created under AMBARI_DIR/ambari-agent/target Ubuntu 14 1)Ambari Repositories Base URL http://public-repo-1.hortonworks.com/ambari/ubuntu14/2.x/updates/2.4.2.0 Repo File http://public-repo-1.hortonworks.com/ambari/ubuntu14/2.x/updates/2.4.2.0/ambari.list Tarball md5 http://public-repo-1.hortonworks.com/ambari/ubuntu14/2.x/updates/2.4.2.0/ambari-2.4.2.0-ubuntu14.tar.gz 2)HDP 2.5 Repositories Repo File http://public-repo-1.hortonworks.com/HDP/ubuntu14/2.x/updates/2.5.3.0/hdp.list Tarball md5 http://public-repo-1.hortonworks.com/HDP/ubuntu14/2.x/updates/2.5.3.0/HDP-2.5.3.0-ubuntu14-deb.tar.gz Tarball md5 http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.21/repos/ubuntu14/HDP-UTILS-1.1.0.21-ubuntu14.tar.gz


原文轉載至:http://www.linuxdiyf.com/linux/28883.html

注意!

本站转载的文章为个人学习借鉴使用,本站对版权不负任何法律责任。如果侵犯了您的隐私权益,请联系我们删除。



 
粤ICP备14056181号  © 2014-2021 ITdaan.com