Archive

Archive for the ‘bashism’ Category

there are time(s) . . .

March 31st, 2011 2 comments

Stefan ist bash Nutzer, ich auch . . . und wir sind beide unabhängig voneinander in die selbe Falle getappt. Wir wollten die Ausgabe von “time” in eine Datei schreiben, dabei hat der naive Anstaz mit Streams umleiten nicht auf Anhieb funktioniert. Eigentlich ganz einfach, die manpage sagt “-o” . . . logisch. Nur tut es dann nicht – WTF? Stefan hats fluchend mit googlen und Foren lesen pobiert, ich hab durch Zufall in der manpage den passenden Hinweis gefunden:

Users of the bash shell need to use an explicit path in order to run the external time command and not the shell builtin variant.

Die bash bringt also ihr eigenes “time” mit das die Funktionalität “-o” nicht bietet . . .
Das Ende der Geschichte:

/usr/bin/time -o /tmp/time.log -p sleep 1

Categories: bashism Tags:

DVB Desktop Widgets

September 30th, 2010 3 comments

Da die offiziellen Widgets des DVB nicht gerade toll benutzbar sind (nur 1 Haltestelle pro Rechner, …), hier mal ein kleiner Einzeiler den man z.B. mit GeekTool oder ähnlichem auf den Desktop werfen kann (siehe Screenshot).

echo "Schneebergstraße"; \
curl -s "http://widgets.vvo-online.de/abfahrtsmonitor/Abfahrten.do?ort=Dresden&hst=SNS" | \
sed 's/\],\[/\
/g;s/\[\[//;s/\]\]//;s/"//g' | awk 'BEGIN {FS=","} {print $3 "\t" $1 "\t" $2}'

Die Codes für die Haltestellen gibt es hier. Notorische Busverpasser können auch im AWK-Teil ein paar Minuten von der verbleibenden Zeit subtrahieren 🙂

Categories: bashism, fun, nerdcore, software Tags:

me is back

September 17th, 2010 1 comment

‘been absent for a while – PTO and stuff

Here are the two major show-stoppers of today which nearly cracked my head:
PTY allocation request failed on channel 0

This occured when I tried to login to a xen domain I just created – took a while to figure out how to solve that but doing a
ssh user@host "/bin/bash -i" (notice the -i for interactive)
helped in accessing the vm and then i could reset the dev’s:
rm -r /dev/ptmx
mknod /dev/ptmx c 5 2
chmod 666 /dev/ptmx
umount /dev/pts
rm -r /dev/pts
mkdir /dev/pts
mount /dev/pts

The second Problem was a erroneous cd-rom drive that stopped working in the middle of the installation process. The wonderfull tool UNetbootin saved the day by providing me with a bootable USB stick in a matter of seconds.

Happy weekend,
Willi

Categories: bashism, software Tags:

Shell Script-Rätselraten

March 12th, 2010 1 comment

Gelegentlich kommt es vor, das großes Rätselraten herrscht, über bestimmt Spitzfindigkeiten von Unix Shells. Sei es das Umbiegen von Deskriptoren oder das Verhalten bei Forks…

Zufällig bin ich gerade auf einen einen sehr schönen Foliensatz gestoßen, der zumindest für die bash abhilfe schafft. Sehr übersichtlich, perfekt zum Nachschlagen.

http://www.cs.uni-potsdam.de/ml/teaching/ws09/rnb/f07.pdf

Categories: bashism, linux, software Tags:

Pwned by error message

August 17th, 2009 6 comments

In the life of an IT-guy you often shake your head and smile while wondering about the current problem or its solution. So after a while not much can realy surprise you. But last week there was such a rare moment.

The problem is an easy one: we could not execute binaries. We have a cluster here with an 32 bit head node and 64 bit compute nodes both x86 Debian GNU Linux. The stuff to be shared comes over a NFS. Heterogenouse clusters have some difficulties because you cannot execute binaries from one architecture on another. This will fail with the error message: “wrong ELF format”.
Our problem first appeared by using the Modules-tool. For testing we executed it directly:

./modulecmd
-bash: ./modulecmd: no such file or directory

But OK, as everybody could see, it was. We checked and changed the rights for users, groups, owndership, parent directories etc. Nothing helped. Next we put an eye on NFS. No miss-configuration or other distinctive feature.
Now we use the bigger weapons and strace the command.

strace ./modulecmd
execve("./modulecmd", ["./modulecmd"], [/* 25 vars */]) = -1 ENOENT (No such file or directory)
dup(2)                                  = 3
fcntl(3, F_GETFL)                       = 0x8002 (flags O_RDWR|O_LARGEFILE)
fstat(3, {st_mode=S_IFCHR|0620, st_rdev=makedev(136, 0), ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7ff4b4700000
lseek(3, 0, SEEK_CUR)                   = -1 ESPIPE (Illegal seek)
write(3, "strace: exec: No such file or dir"..., 40strace: exec: No such file or directory
) = 40
close(3)                                = 0
munmap(0x7ff4b4700000, 4096)            = 0
exit_group(1)                           = ?

Also nothing surprising in here.
The further isolate the root of the behaviour, we code a short “hello world” in C and do compiling and execution and it worked. So here we got the main point which confused us. Self created binaries and all system tools worked properly, just the shared ones or let’s say “not system contained” executables failed.
OK, maybe it’s situated in a deeper level. We installed dash and try to execute it in there. No changes. Slowly but steady we ran out of ideas.
From the strace run we could see that execve backfired. This command overwrites the current forked process with the binary given by a path as an argument and executes it. So maybe the reason belongs to the execution context. We tried

bashism

instead. So WTF? After ignoring some charset ideas in mind, we arrived where we started. An obligatory file on a system executable and our binary shared over NFS told us the wrong ELF type. The installation of ia32-libs fixed the problem finally, but why there was a “no such file of directory” instead of a “wrong ELF type” we still do not know.

You are welcome to give us solution or hints.

Categories: bashism, linux Tags: , ,

scp für viele kleine Dateien

August 8th, 2009 1 comment

Immer benutze ich folgendes hilfreiches Kommando, welches elegant und schnell viele kleine Dateien per ssh kopiert:

tar cf – <dir_to_copy> | ssh <bar@foo> “tar xpf – -C <dest_dir>”

Das Ergebnis befindet sich dann auf foo in <dest_dir>/<dir_to_copy>.

Warum nimmt man da nicht scp? Scp hat die bei vielen kleinen Dateien unangenehme Eigenschaft, jede Datei einzeln zu puffern und dann zu übertragen. Das kann selbst bei gewaltigen Bandbreiten dann zu übertragungsraten von wenigen kB/s führen, was schon mal nervig sein kann.

Durch die Benutzung von tar wird das ganze Verzeichnis als einzelner Stream in großen Blöcken übertragen wie man sich das wünscht.

Categories: bashism, linux, management, software Tags:

compiling from sources

June 26th, 2009 No comments

I’ve been hacking around the last 4 days trying to build up a stack of 5 hpc tools all depending on each other each with a special set of features enabled – pure horror. After switching between cutting edge versions and wildly interchanging compiler-suites all with different non-positive outcome I think I’m finally done. Man vs machine 1:0 – gotcha 😉

Here 2 of the more interesting snippets I had to create to get things up an running:

# add missing string.h
echo "#include  string.h" >swap.c
cat ./SRC/SRC/mpo_LogCube3.cc >>swap.c
mv swap.c ./SRC/SRC/mpo_LogCube3.cc
# fix wrong marmotcc (missing scalasca-lib)
sed -e 's/-lcube3/-lcube3 -lsc.z/g' `which marmotcc` >marmotcc
chmod +x marmotcc
mv ./marmotcc `which marmotcc`

I had to use dos2unix as one script had windows coding and so my shell was complaining about not knowing /bin/bash^M and so on. I also wrote a script creating ./configure lines with all possible combinations of a set of flags to test which combination actually works . . .
Sometimes I’d really like those days back were a “cc hello.c -o HelloWorld” was all that was needed.

BNW

Categories: bashism, c/c++, linux, software Tags: , , ,

It’s about time

June 17th, 2009 1 comment

The tiny tool time everybody should know, measures execution time of a given command. For example:

time echo "test"

prints time information to the console. But what if you want to collect these statistics in a file? Linux man page say “-o FILE” is the way to do so. This will look like

time  -o file.log echo "test"

This FAILS not only on my machine with the error message “bash -o: unknown command” or equal. The key is to give the full path:

/usr/bin/time -o file.log echo "test"

The problem is, that “time” is used for different things. This leads to another cute tool called type. type gives information about the kind of a command.

type -a time
 
time is a shell keyword
time is /usr/bin/time

As you can see “time” not only means the command. If somebody knows more about the “Why the hell is this so?”, feel free to post 😀

Categories: bashism, linux, software Tags: , , , ,

DOS – The Default Option Set Series #1: GNU grep

May 27th, 2009 3 comments

With this post I will start an article series about default options for popular shell tools. And I would like to invite all of you to participate in this process to find a profound default option set for these tools. I will provide my current DOS as a discussion base and extend these by user comments.

In this first DOS its all about GNU grep.

Overview:

Some fundamental information about the considered tool.

Name: GNU grep
Version: 2.5.3
URL: http://www.gnu.org/software/grep/

How to set default options:

This section will describe how to set the Default Option Set. For grep its the environmental variable GREP_OPTIONS.

Default Option Set:

In this section comes a descriptive list of the options.

  • ignore binary files
    --binary-files=without-match
  • use colored output in terminals
    --color=auto
  • ignore special files like fifos, char and block devices
    --devices=skip
  • exclude patch *.rej and *.orig files
    --exclude='*.rej'
    --exclude='*.orig'
  • exclude common scm directories
    --exclude-dir=.git
    --exclude-dir=.svn
    --exclude-dir=CVS
    --exclude-dir=patches
    --exclude-dir=.pc
  • exclude build directories
    --exclude-dir=.libs
    --exclude-dir=.deps

Copy’n’Paste:

And here is a ready-to-use version of the option set.

export GREP_OPTIONS="--binary-files=without-match \
--color=auto \
--devices=skip \
--exclude='*.rej' \
--exclude='*.orig' \
--exclude-dir=.git \
--exclude-dir=.svn \
--exclude-dir=CVS \
--exclude-dir=patches \
--exclude-dir=.pc \
--exclude-dir=.libs \
--exclude-dir=.deps"

Categories: bashism, DOS, linux Tags: , ,

Terminating SSH Sessions

May 14th, 2009 No comments

I think we all know the problem: a ssh session hangs for some reason, either because you are no longer connected to that wifi or because the remote host just crashed.

A more elegant solution than closing your local term (or killing the ssh client) consists of a well hidden feature:
by sequentially pressing the three keys Return ~ . you can exit the client and return to your local prompt.

Categories: bashism, linux, software Tags: , ,