danieldc / advanced-shell-history Goto Github PK

What steps will reproduce the problem?
1. In your "Custom Installation" Tutorial you recommend to prefix the path 
variables with a ~ . 

What is the expected output? What do you see instead?

This should then expand to ${HOME} in the scripts. However, tilde-expansion 
does not work in your scripts, since you always expand the variables inside 
double-quotes. This will give file-not-found errors. Combined with the previous 
bug I filed (on using exit instead of return) this bug will lead to a unusable 
shell environment when following your tutorial.

In MacOS under bash it runs without errors, but produces the following history 
log:

1,2,3,4,5,6,7,8,9,10,11
1,2,3,4,5,6,7,8,9,10,11
1,2,3,4,5,6,7,8,9,10,11
....

> What steps will reproduce the problem?
1. curl 'http://advanced-shell-history.googlecode.com/files/ash-0.1.r92.tar.gz' 
| tar -xzvf - -C /tmp
2. cd /tmp/ash-0.1/ && make
3. ./src/ash_query

> What is the expected output? What do you see instead?
Provided use case should execute program without producing SEGFAULT. 

> What version of the product are you using? On what operating system?
I tried both ash-0.1.r92 and trunk version using different computers with 
Gentoo and Ubuntu with kernel versions 3.4.4 and 3.0.0 respectively. Every 
combination (including system wide installation) yields the same result and 
backtrace. 

> Please provide any additional information below.
% gdb ./src/ash_query
GNU gdb (Ubuntu/Linaro 7.3-0ubuntu2) 7.3-2011.08
Copyright (C) 2011 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "i686-linux-gnu".
For bug reporting instructions, please see:
<http://bugs.launchpad.net/gdb-linaro/>...
Reading symbols from /tmp/ash-0.1/src/ash_query...done.
(gdb) run
Starting program: /tmp/ash-0.1/src/ash_query 
[Thread debugging using libthread_db enabled]

Program received signal SIGSEGV, Segmentation fault.
0xb7f22806 in 
std::__detail::_List_node_base::_M_hook(std::__detail::_List_node_base*) () 
from /usr/lib/i386-linux-gnu/libstdc++.so.6
(gdb) backtrace
#0  0xb7f22806 in 
std::__detail::_List_node_base::_M_hook(std::__detail::_List_node_base*) () 
from /usr/lib/i386-linux-gnu/libstdc++.so.6
#1  0x080ce055 in _M_insert (__position=..., __x=<optimized out>, 
this=<optimized out>) at /usr/include/c++/4.6/bits/stl_list.h:1516
#2  push_back (__x=<optimized out>, this=0x80ff870) at 
/usr/include/c++/4.6/bits/stl_list.h:988
#3  flag::Flag::Flag (this=0x80ff73c, ln=0x80e6fdb "database", sn=100 'd', 
ds=0x80e6fbe "A history database to query.", ha=true)
    at flags.cpp:178
#4  0x080ce561 in flag::StringFlag::StringFlag (this=0x80ff73c, ln=0x80e6fdb 
"database", sn=100 'd', val=0x80ff724, dv=0x0, 
    ds=0x80e6fbe "A history database to query.") at flags.cpp:296
#5  0x0804b7a5 in __static_initialization_and_destruction_0 
(__initialize_p=<optimized out>, __priority=<optimized out>) at ash_query.cpp:44
#6  _GLOBAL__sub_I__Z7displayRSoRKSt3mapISsSsSt4lessISsESaISt4pairIKSsSsEEERS4_ 
() at ash_query.cpp:205
#7  0x080dde82 in __libc_csu_init ()
#8  0xb7d4a0aa in __libc_start_main (main=0x804b090 <main(int, char**)>, 
argc=1, ubp_av=0xbffff684, init=0x80dde30 <__libc_csu_init>, 
    fini=0x80ddea0 <__libc_csu_fini>, rtld_fini=0xb7feeba0, stack_end=0xbffff67c) at libc-start.c:185
#9  0x0804bf25 in _start ()

I use kubuntu 13.10 on two computers: a desktop workstation and a laptop.
On the desktop computer everything works fine: installation was smooth. I set 
the environment variable ASH_LOG_BIN through the KDE startup folder 
"/home/username/.kde/env/ash_startup.sh" to the value:
export ASH_LOG_BIN=/usr/local/bin/_ash_log.py

In .bashrc then I source the file /usr/lib/advanced_shell_history/bash:
source /usr/lib/advanced_shell_history/bash

The desktop computer is fine with this setup and works perfectly, so that I can 
query the command history.

On my laptop I reproduced the same environment, with the same set of files, but 
the python script does not work. The error is the one reported below

username@host:~$ source /usr/lib/advanced_shell_history/bash
Traceback (most recent call last):
  File "/usr/local/bin/_ash_log.py", line 222, in <module>
    sys.exit(main(sys.argv))
  File "/usr/local/bin/_ash_log.py", line 197, in main
    session_id = Session().Insert()
  File "/usr/local/bin/_ash_log.py", line 81, in __init__
    'host_ip': unix.GetHostIp(),
  File "/usr/local/lib/advanced_shell_history/unix.py", line 90, in GetHostIp
    for addresses in _ParseIfconfig().itervalues():
  File "/usr/local/lib/advanced_shell_history/unix.py", line 71, in _ParseIfconfig
    for line in _GetIfconfig():
  File "/usr/local/lib/advanced_shell_history/unix.py", line 56, in _GetIfconfig
    except Error:
NameError: global name 'Error' is not defined
Advanced Shell History enabled: session 
username@host:~$

On the other side, using the C binary in 
"/home/username/.kde/env/ash_startup.sh":
export ASH_LOG_BIN=/usr/local/bin/_ash_log

I have no problems and the software works perfectly.

What steps will reproduce the problem?
1. Open a shell
2.
3.

What is the expected output? What do you see instead?

expected output:
Advanced Shell History enabled: session #id#

error in a python file:
username@host:~$ source /usr/lib/advanced_shell_history/bash
Traceback (most recent call last):
  File "/usr/local/bin/_ash_log.py", line 222, in <module>
    sys.exit(main(sys.argv))
  File "/usr/local/bin/_ash_log.py", line 197, in main
    session_id = Session().Insert()
  File "/usr/local/bin/_ash_log.py", line 81, in __init__
    'host_ip': unix.GetHostIp(),
  File "/usr/local/lib/advanced_shell_history/unix.py", line 90, in GetHostIp
    for addresses in _ParseIfconfig().itervalues():
  File "/usr/local/lib/advanced_shell_history/unix.py", line 71, in _ParseIfconfig
    for line in _GetIfconfig():
  File "/usr/local/lib/advanced_shell_history/unix.py", line 56, in _GetIfconfig
    except Error:
NameError: global name 'Error' is not defined
Advanced Shell History enabled: session 
username@host:~$

What version of the product are you using? On what operating system?
advanced shell history version from the repository trunk folder at revision 165
kubuntu 13.10 fully up to date

Please provide any additional information below.
Please contact me if you need more details on this

The troubleshooting HOWTO page 
(https://code.google.com/p/advanced-shell-history/wiki/HOWTO_Troubleshoot) 
currently says that logging can be disabled by setting the ASH_DISABLE 
environment variable to any non-empty string. However, the actual env variable 
that will disable logging is ASH_DISABLED (as verified by testing and by 
reading the code).

What steps will reproduce the problem?
1. On my linux box, I tried to follow the instructions at 
https://code.google.com/p/advanced-shell-history/wiki/HOWTO_Install. The curl 
command through the tar work fine.
2. The instructions say to cd to /tmp/ash-src, I see /tmp/ash-0.5, and 
/tmp/ash-0.5/src, so already the directions are unclear.
3. Trying first /tmp/ash-0.5, make build produces:

sed -i -e "/^VERSION :=/s/:= .*/:= 0.5.r .rconflicts: .rSkipped/" 
python/Makefile src/Makefile

Compiling source code...
make[1]: Entering directory `/tmp/ash-0.5/python'
sed -i -e "s:^__version__ = .*:__version__ = '0.5.r .rconflicts: .rSkipped':" 
_ash_log.py advanced_shell_history/*.py ash_query.py
sed: -e expression #1, char 55: unknown option to `s'
make[1]: *** [version] Error 1
make[1]: Leaving directory `/tmp/ash-0.5/python'
make: *** [build_python] Error 2

make build_python produces exactly the same output.

What is the expected output? What do you see instead?

See above.

What version of the product are you using? On what operating system?

I guess ash-0.5. Linux.

Please provide any additional information below.

The problem:

I want to know which actual binary a command used.  However, sometimes the 
'command' was actually a shell alias or a shell function.  The best way to know 
is to look at the output of the shell builtin 'type'.

The problem with this, is that it's unsafe to evaluate the previous command 
within the shell code using type, because it potentially re-executes the 
command.


For example:

I entered the command:
  $ less ` which installer.py `

I am interested in 'less' and I'm interested in 'which' - I want to know which 
binaries I was actually using.  The actual previous command, as saved in the 
history db is: 'less ` which installer.py `'

The naive approach is to simply invoke 'eval type "${previous_command}"', which 
will evaluate to: eval type "less /some/path/to/installer.py"

This gives me information for 'less', but it fails to give me information for 
'which' - instead I get information for /some/path/to/installer.py.


What is likely needed is a simple binary helper that takes a raw command as 
input and parses it to identify the command parts of it ('less' and 'which') 
and strip out the other parts.

For example:

  $ ash_helper -c 'less ` which installer.py `'
  less which

  $ builtin type $( !! )
  less is /usr/bin/less
  which is /usr/bin/which


Having this utility makes it possible to effectively filter garbage commands.  
For example, if I accidentally entered 'sl' instead of 'ls' - I might not care 
to see that in my history.  However, if I entered 'ls foo' and foo does not 
exist, I'll get an error code for the command.  I probably want to see this in 
my history.  Since both cases have an error exit code, I can't filter by exit 
code alone.  I need to know if the invoked binary exists or not to filter 
garbage commands.

Include the uname -a output in the sessions table.

This will involve having existing tables altered to add the new column as well 
as having the new column created on new databases.

What steps will reproduce the problem?
1. use ash_query -f aligned ...

What is the expected output? What do you see instead?

Having the spaces at the end of the line serves no useful purpose - they should 
be stripped before inserting the value.

So far this has been seen in zsh, but not confirmed in bash.

What steps will reproduce the problem?
1. source the shell code from a new terminal
2. open a screen session or a subshell
3. enter commands and exit the screen session or subshell
4. enter more commands in the original parent shell

What is the expected output? What do you see instead?

I expect all history to be logged.  In practice, I see constraint violations 
inserting commands since the key uses only the command number and session id.  
Since subshells and screen sessions inherit these values, the constraints are 
not exclusive enough.

We should probably also include the shell PID or possibly the command itself

Please use labels and text to provide additional information.

In common - this should be fixed!

Is there a script to migrate an existing bash history into ASH?

using ash 0.5.r .r158 on arch linux i get the following error message:

/bin/hostname: invalid option -- 'I'
Try 'hostname --help' or 'hostname --usage' for more information.

changing the GetHostIp() function in unix.py (line 49) from "hostname -I" to 
"hostname -i" fixes the problem for me...

What steps will reproduce the problem?
1. install and enable ash
2. from within an ash session run "bash -i"

What is the expected output? What do you see instead?

I expected to get a working ash session. Instead every time I run a command I 
get "ash::precmd: command not found".

This looks like it's due to the line

[[ -n "${ASH_SESSION_ID}" ]] && return

in /usr/lib/advanced_shell_history/bash, which means that when the 
ASH_SESSION_ID variable is inherited across the the bash invocation it causes 
the library to drop out and the function definitions to not be available.

Looking at the code and the bash manpage I reckon what would work as a 
workaround would be to make only some of the library file conditional on an 
existing session, then explicitly check to see if the ash functions existed and 
if not, define them.

What version of the product are you using? On what operating system?

This was a fresh subversion checkout of the code running on an Ubuntu Utopic 
desktop install.

Please provide any additional information below.

Bash version:
GNU bash, version 4.3.30(1)-release (x86_64-pc-linux-gnu)
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>

This is free software; you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

ash version:
0.7.r .r165

What steps will reproduce the problem?
1.install fedora 21 xfce
2.install gcc-c++
3.follow install guide

What is the expected output? What do you see instead?
Expected output is correct compiling of program when executing "make build". I 
see instead self-explanatory Error when compiling database.cpp, that states 
that getpid() is not defined.

What version of the product are you using? On what operating system?
Latest version as of 2015-APR-22 (ash-0.5), operating system is:
Fedora 21 (Linux [...] 3.19.3-200.fc21.x86_64 #1 SMP Thu Mar 26 21:39:42 UTC 
2015 x86_64 x86_64 x86_64 GNU/Linux)

Please provide any additional information below.
adding these 2 lines to database.cpp:
#include <sys/types.h>
#include <unistd.h>
as recommended here: http://man7.org/linux/man-pages/man2/getpid.2.html
and executing "make build" again compiles program successfully.

thanks for the program!! :)
hope this helps others
Max

What steps will reproduce the problem?
1.Install liquidprompt (or some other command that uses PROMPT_COMMAND)
2.Install advanced-shell-history
3.

What is the expected output? What do you see instead?
Ash should automatically save PROMPT_COMMAND, ie, Liquidprompt should work with 
ash as advertised, but ash overwrites PROMPT_COMMAND

What version of the product are you using? On what operating system?
1.55

Please provide any additional information below.

I can fix it by manually setting ASH_PROMPT_COMMAND prior to starting ash, ie:

export ASH_PROMPT_COMMAND=$PROMPT_COMMAND
source /usr/lib/advanced_shell_history/bash

What steps will reproduce the problem?
1. On an up to date Fedora 17 64-bit server. 
2. Download and extract ash
3. Try to compile it (make build_c)



What is the expected output? What do you see instead?
Expected successful compile, instead compile failed and I was greeted with 
errors, including "database.cpp:192:28: error: ‘getpid’ was not declared in 
this scope"


What version of the product are you using? On what operating system?
Linux Fedora 17 64-bit. 
ash-0.5


Please provide any additional information below.

Detailed error (make -d build_c)

...
Invoking recipe from Makefile:41 to update target `database.o'.
make[1]: Entering directory `/home/brad/Downloads/ash-0.5/src'
g++ -c -g -Wall -DASH_VERSION="\"0.5.r .rconflicts: .rSkipped\"" -ansi 
-pedantic -O2 -o database.o database.cpp -lrt
Putting child 0x183c560 (database.o) PID 10540 on the chain.
Live child 0x183c560 (database.o) PID 10540 
database.cpp: In function ‘void ash_sleep()’:
database.cpp:192:28: error: ‘getpid’ was not declared in this scope
Reaping losing child 0x183c560 PID 10540 
make[1]: *** [database.o] Error 1
Removing child 0x183c560 PID 10540 from chain.
make[1]: Leaving directory `/home/brad/Downloads/ash-0.5/src'
Reaping losing child 0x2455a20 PID 10537 
make: *** [build_c] Error 2
Removing child 0x2455a20 PID 10537 from chain.

I've come up with a collection of patches that resolve a number of the issues 
reported against this project. I'm creating this report as a catch-all to 
collect these in one place.

At this point I'm starting to think that moving this project to github would be 
a good idea, though without being able to modify these pages it would be hard 
for existing users to get from here to there.

Simon

What steps will reproduce the problem?
make build

What version of the product are you using? On what operating system?
ash 0.5
GCC 4.8.4
Ubuntu 14.04.2 LTS

Please provide any additional information below.
To fix:
Add 
"""
#include <unistd.h>    /* for getpid */
"""

to database.cpp

As noted in a user comment on the wiki:
BTW, I got a bunch of errors in "database.cc", here's the log of things I did

dhcp-172-27-196-220:Downloads yaroslavvb$ svn checkout 
http://advanced-shell-history.googlecode.com/svn/branches/rb-0.1/ /tmp/ash-src 
A /tmp/ash-src/LICENSE A /tmp/ash-src/files A /tmp/ash-src/files/usr A 
/tmp/ash-src/files/usr/share A /tmp/ash-src/files/usr/share/man A 
/tmp/ash-src/files/usr/share/man/man1 A 
/tmp/ash-src/files/usr/share/man/man1/advanced_shell_history.1.gz A 
/tmp/ash-src/files/usr/lib A /tmp/ash-src/files/usr/lib/advanced_shell_history 
A /tmp/ash-src/files/usr/lib/advanced_shell_history/zsh A 
/tmp/ash-src/files/usr/lib/advanced_shell_history/common A 
/tmp/ash-src/files/usr/lib/advanced_shell_history/bash A 
/tmp/ash-src/files/usr/local A /tmp/ash-src/files/usr/local/bin A 
/tmp/ash-src/files/etc A /tmp/ash-src/files/etc/ash A 
/tmp/ash-src/files/etc/ash/ash.conf A /tmp/ash-src/files/etc/ash/queries A 
/tmp/ash-src/TODO A /tmp/ash-src/src A /tmp/ash-src/src/ash_query.cpp A 
/tmp/ash-src/src/flags.hpp A /tmp/ash-src/src/fetch_sqlite.sh A 
/tmp/ash-src/src/formatter.cpp A /tmp/ash-src/src/database.cpp A 
/tmp/ash-src/src/ash_query.hpp A /tmp/ash-src/src/config.cpp A 
/tmp/ash-src/src/formatter.hpp A /tmp/ash-src/src/database.hpp A 
/tmp/ash-src/src/config.hpp A /tmp/ash-src/src/ash_log.cpp A 
/tmp/ash-src/src/README A /tmp/ash-src/src/util.cpp A 
/tmp/ash-src/src/queries.cpp A /tmp/ash-src/src/command.cpp A 
/tmp/ash-src/src/logger.cpp A /tmp/ash-src/src/ash_log.hpp A 
/tmp/ash-src/src/queries.hpp A /tmp/ash-src/src/util.hpp A 
/tmp/ash-src/src/unix.cpp A /tmp/ash-src/src/command.hpp A 
/tmp/ash-src/src/session.cpp A /tmp/ash-src/src/logger.hpp A 
/tmp/ash-src/src/TODO A /tmp/ash-src/src/queries.l A 
/tmp/ash-src/src/session.hpp A /tmp/ash-src/src/unix.hpp A 
/tmp/ash-src/src/flags.cpp A /tmp/ash-src/src/Makefile A /tmp/ash-src/man A 
/tmp/ash-src/man/ash_query.1 A /tmp/ash-src/man/ash_log.1 A /tmp/ash-src/README 
A /tmp/ash-src/Makefile Checked out revision 110. dhcp-172-27-196-220:Downloads 
yaroslavvb$ cd /tmp/ash-src dhcp-172-27-196-220:ash-src yaroslavvb$ ls LICENSE   
Makefile    README   TODO    files   man     src dhcp-172-27-196-220:ash-src yaroslavvb$ 
make sed -i -e "/^VERSION :=/s/:= ./:= 0.1.r110/" src/Makefile

Compiling source code... ./fetch_sqlite.sh Inspecting 
http://sqlite.org/download.html to guess latest version... Archive: 
sqlite-amalgamation-3071000.zip

creating: sqlite-amalgamation-3071000/
inflating: sqlite-amalgamation-3071000/sqlite3.h inflating: 
sqlite-amalgamation-3071000/shell.c inflating: 
sqlite-amalgamation-3071000/sqlite3ext.h inflating: 
sqlite-amalgamation-3071000/sqlite3.c
gcc -DSQLITE_OMIT_LOAD_EXTENSION -DSQLITE_THREADSAFE=0 -c sqlite3.c g++ -c -g 
-Wall -DASH_VERSION="\"0.1.r110\"" -ansi -pedantic -O2 -lrt -o ash_log.o 
ash_log.cpp i686-apple-darwin10-g++-4.2.1: -lrt: linker input file unused 
because linking not done g++ -c -g -Wall -DASH_VERSION="\"0.1.r110\"" -ansi 
-pedantic -O2 -lrt -o command.o command.cpp i686-apple-darwin10-g++-4.2.1: 
-lrt: linker input file unused because linking not done g++ -c -g -Wall 
-DASH_VERSION="\"0.1.r110\"" -ansi -pedantic -O2 -lrt -o config.o config.cpp 
i686-apple-darwin10-g++-4.2.1: -lrt: linker input file unused because linking 
not done g++ -c -g -Wall -DASH_VERSION="\"0.1.r110\"" -ansi -pedantic -O2 -lrt 
-o database.o database.cpp database.cpp: In function ‘void ash_sleep()’: 
database.cpp:200: error: ‘CLOCK_MONOTONIC’ was not declared in this scope 
database.cpp:200: error: ‘clock_gettime’ was not declared in this scope 
database.cpp:220: error: ‘CLOCK_MONOTONIC’ was not declared in this scope 
database.cpp:220: error: ‘clock_nanosleep’ was not declared in this scope 
database.cpp:256: error: ‘CLOCK_MONOTONIC’ was not declared in this scope 
database.cpp:256: error: ‘clock_gettime’ was not declared in this scope 
make1?: [database.o] Error 1 make: build? Error 2 dhcp-172-27-196-220:ash-src 
yaroslavvb$

What steps will reproduce the problem?

1. User sets $ASH_CFG wrong.
2. Sourcing ~/.ash/lib/bash.
3. Shell will be terminated.
   (If this happens in .bashrc - then the user can not connect to the machine, as the shell is terminated on login)


What is the expected output? What do you see instead?

Sourcing should end, and send correct return code (1). 

This can be achieved with using || return 1 (and similar) instead of || exit 1.

What steps will reproduce the problem?
1. source /usr/lib/advanced_shell_history/bash
2. source /usr/lib/advanced_shell_history/bash

What is the expected output? What do you see instead?

I expect to see no error messages, only the MOTD again, if in use.

Instead I see:
bash: ash::end_session: readonly function
bash: ash::log: readonly function
bash: PROMPT_COMMAND: readonly variable
bash: HISTCONTROL: readonly variable
bash: HISTTIMEFORMAT: readonly variable
bash: ash::begin_session: readonly function
bash: ash::precmd: readonly function
bash: ash::last_command: readonly function

For developing this code, it would be nice to see important debugging messages 
right away, without having to open the log file.  However, I would like to keep 
the messages in the log file as well.

I want to see a new config options: ASH_CFG_LOG_STDERR_LEVEL which sets a 
logging visibility threshold that determines whether a message is logged to 
stderr or not.

This is in addition to the logging threshold that determines whether a message 
is logged to file.

Ideally, I would set my to-file threshold to be something like INFO and my 
to-stderr level to be something like WARNING.  That way, while using a 
developmental release I can see warnings and errors right away but while also 
maintaing a more detailed set of debugging info in a file.


The problem with the existing code is that the Logger inserts some data into a 
ofstream and then returns it to the caller.  At that point the caller can 
insert a message (or messages) at will.  The Logging code loses control of the 
ostream.

What is needed is some kind of proxy ostream that contains two ostreams - one 
open to the file and the other open to stderr.  Instead of returning the 
ostream to the file, we would return this proxy and whenever something is 
inserted, we just check the various logging thresholds to determine whether to 
insert the message into the appropriate streams.

Request:

Add a formatting option to ash_query that groups duplicate entries in the 
leftmost columns.

For example, if I 'select session_id, command from commands order by 
session_id, id', I would like to see all the session_id printed only once for 
each list of session commands.

Instead of this:

session_id  command
----------  ------------------------
1           ls
1           cd /
1           ls
1           ssh root@some_host
1           exit
2           ssh me@my_other_host
2           exit

I want to see something like this:

session_id
    command
1
    ls
    cd /
    ls
    ssh root@some_host
    exit

2
    ssh me@my_other_host
    exit

Finally, there is no reason to limit this to only the leftmost column.  It 
should auto-group repeated values, starting at the leftmost column and continue 
until it hits a column where grouping values is infrequent.