6 Comments

  1. k says:

    April 25, 2013 at 3:21 pm

    Seems like a good idea, although I have two comments about your script:

    1) It doesn’t remove ${LIST} at the end.
    2) It doesn’t handle slotted packages.

    By only picking out column two from the output, it missed, rather ironically given your previous post, that my machine wanted both versions of swig:
    NS dev-lang/swig 1.3.40-r2 [1.3.40-r1]
    U dev-lang/swig 2.0.9 [1.3.40-r1]

    Your script emerged dev-lang/swig-2.0.9 twice.

  2. d says:

    May 1, 2013 at 10:54 am

    Could you accomplish the same thing without parsing emerge output text by running with –keep-going? If I’m right, Portage would gracefully skip whatever packages it couldn’t build due to having their dependencies built (as binpkgs) but not merged. If some packages had build errors it would skip those and their dependents gracefully as well.

    I am interested in this approach of building a round of binpkgs and doing a round of merges because I prefer to keep my / and /usr mounted ro most of the time, but I have to remount them rw for the whole duration of a traditional set of builds because merges take place throughout. The solution I’m particularly curious to test is to branch the LVM volume(s) backing / and /usr, build a bunch of packages with –buildpkg (not only!), and then discard the LVM branch and merge the binpkgs (which you stored on another volume, of course) all at once. If that works, step two is to mount the RW LVM branch only in an LXC container where the build is done; that way the live system’s / and /usr are only mounted read-write for a minute or two at the end of the process, and the whole batch of packages is merged to it (or not) atomically. There’s probably some clever trick you could add to this (more LXC?) to ensure that only Portage can write to / and /usr while they are mounted rw…

    Also, a few friendly suggestions regarding your script:
    1. sh style almost always omits semicolons; the principal exception is to put for/do, if/then, etc. on the same line by separating them with a semicolon:
    for a in foo bar baz ; do
    ...
    done

    case statements are another exception.

    2. Skip the temporary file and do:
    for PACKAGE in $(emerge -puDN --color=n --columns --quiet=y world | awk '{print $2}') ; do
    If you think that line looks too long you can also assign the string to LIST instead of storing it in a file:
    LIST="$(emerge -puDN --color=n --columns --quiet=y world | awk '{print $2}')"
    for PACKAGE in ${LIST} ; do

    If you prefer to keep the temporary file, you should be sure to quote its filename in case mktemp returns a path with spaces (or whatnot) in it:
    LIST="$(mktemp)"
    emerge -puDN --color=n --columns --quiet=y world | awk '{print $2}' > "${LIST}"
    for PACKAGE in $(cat "${LIST}")

    Proper use of quotes in shell scripts is an extremely complicated topic.

    3. echo is preferred over printf if you’re not doing any formatting; you can suppress the trailing newline by passing the -n switch.

    4. Consider cut instead of awk for picking out columns unless you want to do fancy things like format the output, i.e.:
    awk '{print "=$2-$3"}'

    5. When testing the success/failure of simple commands, it’s usually clearer to write the command itself as the conditional:
    if emerge -uN --quiet-build --quiet=y --buildpkgonly ${PACKAGE} ; then
    echo ok
    else
    echo failed
    fi

    Your command is long enough here that running it and then testing $? is quite reasonable too.

    6. Here’s how I might write the whole thing, although I don’t have a Gentoo system in front of me so I’m not positive I’m parsing the emerge output properly:
    #!/bin/sh
    emerge -puDN --color=n --columns --quiet=y world \
    | while read HEAD PACKAGE VERSION TAIL ; do
    echo -n "building =${PACKAGE}-${VERSION} ... "
    emerge -uN --quiet-build --quiet=y --buildpkgonly "=${PACKAGE}-${VERSION}" \
    && echo ok \
    || echo failed
    done

  3. says:

    May 7, 2013 at 10:51 am

    I don’t think the –keep-going keeps going in this case. I tried that earlier on, and found that I continuously had to remove the packages that wouldn’t build in order to build something (iow, Portage stopped during dependency check). Hence the per-package approach above.

    Thanks for the updates on style and such, I’ll incorporate (most of) those in my script ;) I’m generally more inclined to make longer scripts, but more readable to users that aren’t all that familiar with scripting.

  4. says:

    May 14, 2013 at 11:18 am

    Great idea. You’re trying to fix one of the major shortcomings of Gentoo, that it’s not possible to set/update configurations immediately, although rarely mentioned. (People only say “you have to compile all.” It’s true, but what’s concerned here is bigger, because it’s human task and can’t be automated.)

    The idea of Mr/Ms “d”, –keep-going (or another option) in fact __has to be supporetd__ by emerge (1), rather than relying on ad hoc hack of breaking into pieces. It’d also be nice if emerge (1) save tarballs in some temporary directory (e.g. /tmp/buildpkg/YYYY-MM-DD ?), and later merges be automatically done from them, and delete them after merge.

    BTW I’d first merge already built tarballs first with -1 = –oneshot, instead of emerging “world” in one big step. (I haven’t tried it yet though.)

    Thank you very much. =)

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>