一半君的总结纸

听话只听一半君

run travis ci locally for ss builds and tomato builds

测试 ss builds 和 tomato builds

首先当然得先安装docker

dnf install docker

systemctl start docker
配置本地的 travis ci
# change the image according to the language chosen in .travis.yml
$ docker run -it -u travis quay.io/travisci/travis-ruby /bin/bash

# now that you are in the docker image, switch to the travis user if you did not add -u to docker run
sudo - travis

# 显示所有可用ruby版本
rvm list

# Install a recent ruby (default is 1.9.3)
rvm install 2.3.0
rvm use 2.3.0

# 或者把2.3.0设为默认
rvm --default use 2.3.0

# Install travis-build to generate a .sh out of .travis.yml
cd ~/builds
git clone https://github.com/travis-ci/travis-build.git
cd travis-build
gem install travis
travis # to create ~/.travis
ln -s `pwd` ~/.travis/travis-build
bundle install

# Create project dir, assuming your project is `me/project` on GitHub
cd ~/builds
mkdir me
cd me
git clone https://github.com/me/project.git
cd project
# change to the branch or commit you want to investigate
travis compile > ci.sh
# You most likely will need to edit ci.sh as it ignores matrix and env
bash ci.sh
测试lz的tomato-shadowsocks repo
git clone https://github.com/oglopss/tomato-shadowsocks.git

sed -i 's/git@github.com:/https:\/\/github.com\//' .gitmodules
git submodule update --init --recursive

export TRAVIS_BUILD_DIR=~/build/oglopss/tomato-shadowsocks

# for testing
export SS_VER=v2.6.1

. .travis.sh

#
sudo apt-get install bison flex gperf libncurses5-dev texinfo help2man asciidoc xmlto

一些错误的应对(ss-build)

  • ct-ng里调用wget的时候会看到如下错误,无法下载文件

    [ALL  ]    ERROR: no certificate subject alternative name matches
    [ALL  ]         requested host name `uclibc.org'.
    [ALL  ]    To connect to uclibc.org insecurely, use `--no-check-certificate'.
    

    可以通过添加如下设置解决

    check_certificate = off
    
  • 似乎不影响sed的替换,可忽略

    $ sed -i 's/git@github.com:/https:\/\/github.com\//' .gitmodules
    sed: warning: failed to set default file creation context to system_u:object_r:svirt_sandbox_file_t:s0:c86,c760: Permission denied
    
  • udns bypass testing

    # generate patch
    diff -aur udns-0.4/configure.lib  udns-original/configure.lib  > udns-configure.lib.patch
    
    # apply patch
    patch -p1 < ../udns-configure.lib.patch
    

    copy files into and out of docker container

    sudo docker cp travis:/home/travis/builds/oglopss/tomato-shadowsocks/.travis.yml .
    
    sudo docker cp udns-configure.lib.patch travis:/home/travis/builds/oglopss/tomato-shadowsocks/
    

    using wget inside docker container

    OpenSSL: error:1407742E:SSL routines:SSL23_GET_SERVER_HELLO:tlsv1 alert protocol version
    Unable to establish SSL connection.
    Error on line 101
    
测试advancedTomato编译
mkdir -p ~/builds/oglops
git clone -b fedora https://github.com/oglops/advancedtomato.git
git clone -b v3.4-140 https://github.com/oglops/advancedtomato-gui.git

sudo ln -s ~/builds/oglops/advancedtomato/tools/brcm /opt/brcm
export PATH=$PATH:/opt/brcm/hndtools-mipsel-linux/bin:/opt/brcm/hndtools-mipsel-uclibc/bin

rsync -rpv --ignore-times  ./advancedtomato-gui/*  ./advancedtomato/release/src-rt/router/www/  --exclude .git

# 搜 packages
apt-cache search libncurse5

sudo apt-get install  libncurses5 libncurses5-dev bison flex texinfo libgmp-dev nettle-dev libtool autogen automake libstdc++6 net-tools

cd release/src-rt
make distclean ; rm ~/advancedTomato.txt;  time make V1=RT-N5x-CN- V2=-140 r2z  2>&1 | tee ~/advancedTomato.txt

一些错误(advancedTomato)

  • /home/travis/builds/oglops/advancedtomato/tools/brcm/K26/hndtools-mipsel-uclibc-4.2.4/bin/../lib/gcc/mipsel-linux-uclibc/4.2.4/../../../../mipsel-linux-uclibc/bin/ld: cannot find -lhogweed
    collect2: ld returned 1 exit status
    make[6]: *** [dnsmasq] Error 1
    make[6]: Leaving directory `/home/travis/builds/oglops/advancedtomato/release/src/router/dnsmasq/src'
    

    没有GMP的时候,nettle编译的时候会略过hogweed,所以

    apt-get install libgmp3-dev
    
  • touch sqlite/stamp-h1
    make[5]: Entering directory `/home/travis/builds/oglops/advancedtomato/release/src/router/sqlite'
    CDPATH="${ZSH_VERSION+.}:" && cd . && /bin/bash /home/travis/builds/oglops/advancedtomato/release/src/router/sqlite/missing aclocal-1.15 
    /home/travis/builds/oglops/advancedtomato/release/src/router/sqlite/missing: line 81: aclocal-1.15: command not found
    WARNING: 'aclocal-1.15' is missing on your system.
             You should only need it if you modified 'acinclude.m4' or
             'configure.ac' or m4 files included by 'configure.ac'.
             The 'aclocal' program is part of the GNU Automake package:
             <http://www.gnu.org/software/automake>
             It also requires GNU Autoconf, GNU m4 and Perl in order to run:
             <http://www.gnu.org/software/autoconf>
             <http://www.gnu.org/software/m4/>
             <http://www.perl.org/>
    make[5]: *** [aclocal.m4] Error 127
    make[5]: Leaving directory `/home/travis/builds/oglops/advancedtomato/release/src/router/sqlite'
    

    此错误表明 automake版本过低,不幸的是repo里只有1.11, 这个镜像里的Ubuntu版本太老了

    $ cat /etc/*-release
    DISTRIB_ID=Ubuntu
    DISTRIB_RELEASE=12.04
    DISTRIB_CODENAME=precise
    DISTRIB_DESCRIPTION="Ubuntu 12.04.5 LTS"
    NAME="Ubuntu"
    VERSION="12.04.5 LTS, Precise Pangolin"
    ID=ubuntu
    ID_LIKE=debian
    PRETTY_NAME="Ubuntu precise (12.04.5 LTS)"
    VERSION_ID="12.04"
    

    所以lz只好自己装 automake 1.15

    wget http://ftp.gnu.org/gnu/automake/automake-1.15.tar.gz
    tar xf automake*
    cd automake-1.15
    sh configure --prefix /usr
    make install
    
  • CDPATH="${ZSH_VERSION+.}:" && cd . && /bin/bash /home/travis/builds/oglops/advancedtomato/release/src/router/sqlite/missing autoconf
     cd . && /bin/bash /home/travis/builds/oglops/advancedtomato/release/src/router/sqlite/missing automake-1.15 --foreign
    configure.ac:23: error: possibly undefined macro: AC_PROG_LIBTOOL
          If this token and others are legitimate, please use m4_pattern_allow.
          See the Autoconf documentation.
    make[5]: *** [configure] Error 1
    make[5]: *** Waiting for unfinished jobs....
    Makefile.am:4: error: Libtool library used but 'LIBTOOL' is undefined
    Makefile.am:4:   The usual way to define 'LIBTOOL' is to add 'LT_INIT'
    Makefile.am:4:   to 'configure.ac' and run 'aclocal' and 'autoconf' again.
    Makefile.am:4:   If 'LT_INIT' is in 'configure.ac', make sure
    Makefile.am:4:   its definition is in aclocal's search path.
    make[5]: *** [Makefile.in] Error 1
    make[5]: Leaving directory `/home/travis/builds/oglops/advancedtomato/release/src/router/sqlite'
    

    image里的libtool也特别老

    $ libtool --version
    libtool (GNU libtool) 2.4.2
    Written by Gordon Matzigkeit <gord@gnu.ai.mit.edu>, 1996
    

    不过这不是问题所在,解决方法是

    echo /usr/share/aclocal | sudo tee --append /usr/local/share/aclocal/dirlist
    
    export PATH=$PATH:/usr/local/bin
    

    原因见这里

  • make[5]: Entering directory `/home/travis/builds/oglops/advancedtomato/release/src/router/libnfsidmap'
    CDPATH="${ZSH_VERSION+.}:" && cd . && /bin/bash /home/travis/builds/oglops/advancedtomato/release/src/router/libnfsidmap/missing aclocal-1.13 -I m4
    /home/travis/builds/oglops/advancedtomato/release/src/router/libnfsidmap/missing: line 81: aclocal-1.13: command not found
    WARNING: 'aclocal-1.13' is missing on your system.
             You should only need it if you modified 'acinclude.m4' or
             'configure.ac' or m4 files included by 'configure.ac'.
             The 'aclocal' program is part of the GNU Automake package:
             <http://www.gnu.org/software/automake>
             It also requires GNU Autoconf, GNU m4 and Perl in order to run:
             <http://www.gnu.org/software/autoconf>
             <http://www.gnu.org/software/m4/>
             <http://www.perl.org/>
    make[5]: *** [aclocal.m4] Error 127
    

    似乎还要1.13?

    wget http://ftp.gnu.org/gnu/automake/automake-1.13.4.tar.gz
    tar xf automake-1.13.4
    cd automake-1.13.4
    sh configure --prefix /usr
    make install
    
  • LZ4 headers not found.
    LZ4 library or header not found, using version in src/compat/compat-lz4.*
    checking git checkout... no
    configure: error: route utility is required but missing
    make[4]: *** [openvpn/.conf] Error 1
    make[4]: Leaving directory `/home/travis/builds/oglops/advancedtomato/release/src/router'
    

    说明需要 net-tools

    apt-get install net-tools
    
  • checking for restartable system calls... configure: error: in `/home/travis/builds/oglops/advancedtomato/release/src/router/mysql':
    configure: error: cannot run test program while cross compiling
    See `config.log' for more details
    make[5]: *** [config.status] Error 1
    make[5]: Leaving directory `/home/travis/builds/oglops/advancedtomato/release/src/router/mysql'
    

    这个问题很奇怪,明明 router/Makefile里已经写了 cp -f configure.mipsel configure了,可是有时候还是会看到此错误,问题出在Makefile的这一行, 既然已经在subshell里cp -f了,为何会没有覆盖呢? lz在本地测试时此问题随机出现(但是后来又无法再现了),而在线测试时必出现,这看起来不应该有问题

    在这个错误的上面有

    touch mysql/stamp-h1
       mysql 53/98 ]2;mysql 53/98
    make[5]: Entering directory `/home/travis/advancedtomato/release/src/router/mysql'
    cd . && /bin/bash /home/travis/advancedtomato/release/src/router/mysql/missing --run aclocal-1.10 
    /home/travis/advancedtomato/release/src/router/mysql/missing: line 54: aclocal-1.10: command not found
    WARNING: `aclocal-1.10' is missing on your system.  You should only need it if
             you modified `acinclude.m4' or `configure.in'.  You might want
             to install the `Automake' and `Perl' packages.  Grab them from
             any GNU archive site.
     cd . && /bin/bash /home/travis/advancedtomato/release/src/router/mysql/missing --run automake-1.10 --foreign 
    /home/travis/advancedtomato/release/src/router/mysql/missing: line 54: automake-1.10: command not found
    WARNING: `automake-1.10' is missing on your system.  You should only need it if
             you modified `Makefile.am', `acinclude.m4' or `configure.in'.
             You might want to install the `Automake' and `Perl' packages.
             Grab them from any GNU archive site.
    cd . && /bin/bash /home/travis/advancedtomato/release/src/router/mysql/missing --run autoconf
    aclocal.m4:16: warning: this file was generated for autoconf 2.63.
    You have another version of autoconf.  It may work, but is not guaranteed to.
    If you have problems, you may need to regenerate the build system entirely.
    To do so, use the procedure documented by the package, typically `autoreconf'.
    configure.in:23: warning: AC_LANG_CONFTEST: no AC_LANG_SOURCE call detected in body
    ../../lib/autoconf/lang.m4:193: AC_LANG_CONFTEST is expanded from...
    
    

    虽然不知道为啥,但是lz把automake 1.10还有 autoconf 2.63也给装了

  • make[5]: Entering directory `/home/travis/builds/oglops/advancedtomato/release/src/router/nano'
    CDPATH="${ZSH_VERSION+.}:" && cd . && /bin/bash /home/travis/builds/oglops/advancedtomato/release/src/router/nano/missing aclocal-1.15 -I m4
    configure.ac:27: error: Autoconf version 2.69 or higher is required
    configure.ac:27: the top level
    autom4te: /usr/bin/m4 failed with exit status: 63
    aclocal-1.15: error: echo failed with exit status: 63
    WARNING: 'aclocal-1.15' is probably too old.
             You should only need it if you modified 'acinclude.m4' or
             'configure.ac' or m4 files included by 'configure.ac'.
             The 'aclocal' program is part of the GNU Automake package:
             <http://www.gnu.org/software/automake>
             It also requires GNU Autoconf, GNU m4 and Perl in order to run:
             <http://www.gnu.org/software/autoconf>
             <http://www.gnu.org/software/m4/>
             <http://www.perl.org/>
    make[5]: *** [aclocal.m4] Error 63
    

    看来需要装新版的autoconf

    wget http://ftp.gnu.org/gnu/autoconf/autoconf-2.69.tar.gz 
    tar xf autoconf*
    cd autoconf-2.69
    sh configure --prefix /usr
    make install
    
  • make[5]: Leaving directory `/home/travis/advancedtomato/release/src/router/libncurses'
    cd /home/travis/advancedtomato/release/src-rt/router/tor && CC=mipsel-uclibc-gcc STRIP='mipsel-uclibc-strip' \
    	autoreconf && \
    	CFLAGS="-Os -Wall -DLINUX26 -DCONFIG_BCMWL5 -pipe -DBCMWPA2 -funit-at-a-time -Wno-pointer-sign -mtune=mips32 -mips32   -ffunction-sections -fdata-sections" \
    	LDFLAGS="-ffunction-sections -fdata-sections -Wl,--gc-sections" \
    	./configure --host=mipsel-linux --build=x86_64-linux-gnu --prefix=/usr --with-libevent-dir=/home/travis/advancedtomato/release/src-rt/router/libevent/staged/usr/local \
    		--with-openssl-dir=/home/travis/advancedtomato/release/src-rt/router/openssl --with-zlib-dir=/home/travis/advancedtomato/release/src-rt/router/zlib \
    		--disable-asciidoc --disable-tool-name-check --disable-unittests --enable-systemd=no
    main::scan_file() called too early to check prototype at /usr/bin/aclocal line 603.
    Useless use of /d modifier in transliteration operator at /usr/share/automake-1.10/Automake/Wrap.pm line 60.
    configure.ac:14: require Automake 1.11, but have 1.10
    autoreconf: automake failed with exit status: 1
    make[4]: *** [tor/stamp-h1] Error 1
    

    难道每个automake 版本都要装? lz于是把1.11也装了

  • ../libtool: line 937: X--tag=CC: command not found
    ../libtool: line 937: X--tag=CC: command not found
    ../libtool: line 970: libtool: ignoring unknown tag : command not found
    ../libtool: line 970: libtool: ignoring unknown tag : command not found
    ../libtool: line 937: X--mode=compile: command not found
    ../libtool: line 937: X--mode=compile: command not found
    ../libtool: line 1104: *** Warning: inferring the mode of operation is deprecated.: command not found
    ../libtool: line 1105: *** Future versions of Libtool will require --mode=MODE be specified.: command not found
    ../libtool: line 1104: *** Warning: inferring the mode of operation is deprecated.: command not found
    ../libtool: line 1105: *** Future versions of Libtool will require --mode=MODE be specified.: command not found
    ../libtool: line 1248: Xmipsel-uclibc-gcc: command not found
    ../libtool: line 1248: Xmipsel-uclibc-gcc: command not found
    ../libtool: line 1248: X-DHAVE_CONFIG_H: command not found
    ../libtool: line 1248: X-DHAVE_CONFIG_H: command not found
    ../libtool: line 1248: X-I.: command not found
    ../libtool: line 1248: X-I.: command not found
    ../libtool: line 1248: X-I..: command not found
    ../libtool: line 1248: X-I..: command not found
    ../libtool: line 1248: X-I../include: No such file or directory
    ../libtool: line 1248: X-I../include: No such file or directory
    ../libtool: line 1248: X-I/home/travis/advancedtomato/release/src-rt/router/libogg/include: No such file or directory
    ../libtool: line 1248: X-I/home/travis/advancedtomato/release/src-rt/router/libogg/include: No such file or directory
    ../libtool: line 1248: X-I/home/travis/advancedtomato/release/src-rt/router/libogg/include: No such file or directory
    ../libtool: line 1248: X-I/home/travis/advancedtomato/release/src-rt/router/libogg/include: No such file or directory
    ../libtool: line 1248: X-O20: command not found
    ../libtool: line 1248: X-O20: command not found
    ../libtool: line 1248: X-Wall: command not found
    ../libtool: line 1248: X-Wall: command not found
    ../libtool: line 1248: X-Wextra: command not found
    ../libtool: line 1248: X-Wextra: command not found
    ../libtool: line 1248: X-ffast-math: command not found
    ../libtool: line 1248: X-ffast-math: command not found
    ../libtool: line 1248: X-D_REENTRANT: command not found
    ../libtool: line 1248: X-fsigned-char: command not found
    ../libtool: line 1248: X-D_REENTRANT: command not found
    ../libtool: line 1248: X-Wdeclaration-after-statement: command not found
    ../libtool: line 1248: X-fsigned-char: command not found
    ../libtool: line 1248: X-Os: command not found
    ../libtool: line 1248: X-Wdeclaration-after-statement: command not found
    ../libtool: line 1248: X-Wall: command not found
    ../libtool: line 1248: X-Os: command not found
    ../libtool: line 1248: X-DLINUX26: command not found
    ../libtool: line 1248: X-Wall: command not found
    ../libtool: line 1248: X-DCONFIG_BCMWL5: command not found
    ../libtool: line 1248: X-pipe: command not found
    ../libtool: line 1248: X-DLINUX26: command not found
    ../libtool: line 1248: X-DBCMWPA2: command not found
    ../libtool: line 1248: X-DCONFIG_BCMWL5: command not found
    ../libtool: line 1248: X-funit-at-a-time: command not found
    ../libtool: line 1248: X-Wno-pointer-sign: command not found
    ../libtool: line 1248: X-pipe: command not found
    ../libtool: line 1248: X-mtune=mips32: command not found
    ../libtool: line 1248: X-DBCMWPA2: command not found
    ../libtool: line 1248: X-mips32: command not found
    ../libtool: line 1248: X-funit-at-a-time: command not found
    ../libtool: line 1248: X-Wno-pointer-sign: command not found
    ../libtool: line 1248: X-fPIC: command not found
    ../libtool: line 1248: X-mtune=mips32: command not found
    ../libtool: line 1248: X-ffunction-sections: command not found
    ../libtool: line 1248: X-mips32: command not found
    ../libtool: line 1248: X-fdata-sections: command not found
    ../libtool: line 1248: X-fPIC: command not found
    ../libtool: line 1248: X-DUSE_MEMORY_H: command not found
    ../libtool: line 1248: X-ffunction-sections: command not found
    ../libtool: line 1248: X-fdata-sections: command not found
    ../libtool: line 1248: X-MT: command not found
    ../libtool: line 1248: X-DUSE_MEMORY_H: command not found
    ../libtool: line 1248: Xsmallft.lo: command not found
    ../libtool: line 1248: X-MT: command not found
    ../libtool: line 1248: X-MD: command not found
    ../libtool: line 1248: Xmdct.lo: command not found
    ../libtool: line 1248: X-MP: command not found
    ../libtool: line 1248: X-MD: command not found
    ../libtool: line 1248: X-MF: command not found
    ../libtool: line 1248: X-MP: command not found
    ../libtool: line 1248: X.deps/smallft.Tpo: No such file or directory
    ../libtool: line 1248: X-MF: command not found
    ../libtool: line 1248: X-c: command not found
    ../libtool: line 1248: X.deps/mdct.Tpo: No such file or directory
    ../libtool: line 1301: Xsmallft.lo: command not found
    ../libtool: line 1248: X-c: command not found
    ../libtool: line 1306: libtool: compile: cannot determine name of library object from `': command not found
    make[6]: *** [smallft.lo] Error 1
    make[6]: *** Waiting for unfinished jobs....
    ../libtool: line 1301: Xmdct.lo: command not found
    ../libtool: line 1306: libtool: compile: cannot determine name of library object from `': command not found
    make[6]: *** [mdct.lo] Error 1
    make[6]: Leaving directory `/home/travis/advancedtomato/release/src/router/libvorbis/lib'
    

    搜了个解决办法,不过lz依然摸不着头脑

    export echo=echo
    
  • 当编译出问题时,彻底恢复
    cd ../../; git clean -xfd; git checkout .; cd -
    
    # 或者这样,加括号的意思是在sub-shell里运行,当前目录不会变
    ( cd ../../ && git clean -xfd &&  git checkout . )
    

有个最大问题是travis ci免费用户最大编译时间只允许50min? lz本地测试了下r2z(AIO)版本,不知道这会不会超时

real    41m3.523s
user    49m57.027s
sys     10m51.568s

Some Tips
  • # 安装locate
    sudo apt-get install mlocate
    sudo updatedb
    
    # 但还是不行
    $ sudo addgroup mlocate
    Adding group `mlocate' (GID 1003) ...
    groupadd: failure while writing changes to /etc/group
    addgroup: `/usr/sbin/groupadd -g 1003 mlocate' returned error code 10. Exiting.
    
  • yum whatprovides的等价物是apt-file
    apt-get install apt-file
    apt-file update
    apt-file search hogweed
    
  • 测试中因为pull了ubuntu image,又clone了advancedTomato,而且编译了,所以默认10gb空间一下子就满了

    $ sudo docker info           
    Containers: 1
     Running: 0
     Paused: 0
     Stopped: 1
    Images: 3
    Server Version: 1.12.6
    Storage Driver: devicemapper
     Pool Name: docker-253:0-4989893-pool
     Pool Blocksize: 65.54 kB
     Base Device Size: 10.74 GB
     Backing Filesystem: xfs
     Data file: /dev/loop0
     Metadata file: /dev/loop1
     Data Space Used: 19.03 GB
     Data Space Total: 107.4 GB
     Data Space Available: 14.86 GB
     Metadata Space Used: 14.63 MB
     Metadata Space Total: 2.147 GB
     Metadata Space Available: 2.133 GB
     Thin Pool Minimum Free Space: 10.74 GB
     Udev Sync Supported: true
    
    
    # lz的container名字是tt
    sudo docker inspect --format {{.GraphDriver.Data.DeviceSize}} tt
    10737418240
    

reference:
How to reproduce a travis-ci build environment for debugging
How To Automatically Skip Wget SSL Certificate Check
curl fails inside container on https connection
How to force wget to overwrite an existing file ignoring timestamp?

How to install the locate command?
How to fix “updatedb: can not find group ` mlocate’” on entware?

Re: configure cannot find -lhogweed
cannot find -lhogweed – linksysinfo

installing autotools, autoconf – ubuntu

编译 OpenVPN 及解决相关依赖问题
Docker 常见问题 (FAQ)

Increasing the Base Device Size on Docker Daemon Restart
daemon option (–storage-opt dm.basesize) for increasing the base device size on daemon restart – docker#19123
docker and image size limit
Docker container: Backup and Recovery
X-I.: command not found, and failed to build certain kind of software, what’s wrong?
Re: libtool errors “command not found” and “ignoring unknown tag”

Advertisements

发表评论

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / 更改 )

Twitter picture

You are commenting using your Twitter account. Log Out / 更改 )

Facebook photo

You are commenting using your Facebook account. Log Out / 更改 )

Google+ photo

You are commenting using your Google+ account. Log Out / 更改 )

Connecting to %s

%d 博主赞过: