First steps with Ipython
IPython is a powerful tool to run python code, with special features to make more easy work in Data Science Forensics. In this video we see how to start to use it.
We can use some tools to run python code for DS4n6 (Data Science Forensics). One of them is Ipython.
IPython add some interesting features to the classic Python interpreter
To use IPython you need: - Know the Python language - Install IPython, by example Installing Anaconda
So you can start IPython typing in your terminal: $ ipython You can see the first difference with classic Python interpreter in the input prompt. You will see In [N] instead of >>> The number N allow to use the content of the past inputs and outputs, we will see this later.
The basic operation in IPython is to introduce python commands and run:
print("May the Force be with you!")
May the Force be with you!
2 + 2
4
Getting help
Ipython have syntax highlighting supports autocompletion using Tab key:
"geonosis".capitalize()
'Geonosis'
You can also obtain help adding the ? (question mark) to anything you write:
str.format?
[0;31mDocstring:[0m
S.format(*args, **kwargs) -> str
Return a formatted version of S, using substitutions from args and kwargs.
The substitutions are identified by braces ('{' and '}').
[0;31mType:[0m method_descriptor
History
Ipython stores all historical inputs and outputs in variables. All the inputs introduced are in the list In and the output in the dictionary Out
print(In)
['', 'print("May the Force be with you!")', '2 + 2', '"geonosis".capitalize()', "get_ipython().run_line_magic('pinfo', 'str.format')", 'print(In)']
print(Out)
{2: 4, 3: 'Geonosis'}
Remember the index of In list starts in 1, not in 0.
Also you can use _, __, ___ to get the last outputs:
print(__)
4
Combining with shell
An important feature of Ipython is that you can use shell commands and combine with python.
You can run a shell command using ! (exclamation mark)
!df -h
Filesystem Size Used Avail Use% Mounted on udev 11G 0 11G 0% /dev tmpfs 2.2G 1.4M 2.2G 1% /run /dev/sda2 20G 6.6G 12G 36% / tmpfs 11G 4.0K 11G 1% /dev/shm tmpfs 5.0M 0 5.0M 0% /run/lock tmpfs 11G 0 11G 0% /sys/fs/cgroup lyra:/export/data/evidence 4.0T 1014G 3.0T 26% /mnt/evidence lyra:/export/data/analysis 4.0T 1014G 3.0T 26% /mnt/analysis tmpfs 2.2G 0 2.2G 0% /run/user/1002 tmpfs 2.2G 0 2.2G 0% /run/user/1009 tmpfs 2.2G 0 2.2G 0% /run/user/1010 tmpfs 2.2G 0 2.2G 0% /run/user/1001 tmpfs 2.2G 0 2.2G 0% /run/user/1008 tmpfs 2.2G 0 2.2G 0% /run/user/1007
But the point is that you can assign the result of the shell command to a variable
os_info = !uname -a os_info
['Linux dasha 4.15.0-108-generic #109-Ubuntu SMP Fri Jun 19 11:33:10 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux']
And we can use a python variable in a shell command with the $ or between {} curly brackets:
filetype = 'directory' !file /etc/* | grep {filetype}
/etc/X11: directory /etc/acpi: directory /etc/alternatives: directory /etc/apache2: directory /etc/apm: directory /etc/apparmor: directory /etc/apparmor.d: directory /etc/apport: directory /etc/apt: directory /etc/bash_completion.d: directory /etc/binfmt.d: directory /etc/byobu: directory /etc/ca-certificates: directory /etc/calendar: directory /etc/cloud: directory /etc/console-setup: directory /etc/cron.d: directory /etc/cron.daily: directory /etc/cron.hourly: directory /etc/cron.monthly: directory /etc/cron.weekly: directory /etc/cryptsetup-initramfs: directory /etc/dbus-1: directory /etc/default: directory /etc/depmod.d: directory /etc/dhcp: directory /etc/dnsmasq.d: directory /etc/dnsmasq.d-available: directory /etc/dpkg: directory /etc/fonts: directory /etc/ghostscript: directory /etc/groff: directory /etc/grub.d: directory /etc/gss: directory /etc/init: directory /etc/init.d: directory /etc/initramfs-tools: directory /etc/insserv.conf.d: directory /etc/iproute2: directory /etc/iscsi: directory /etc/kernel: directory /etc/landscape: directory /etc/ld.so.conf.d: directory /etc/ldap: directory /etc/libnl-3: directory /etc/libpaper.d: directory /etc/lighttpd: directory /etc/logcheck: directory /etc/logrotate.d: directory /etc/lvm: directory /etc/lynx: directory /etc/mdadm: directory /etc/modprobe.d: directory /etc/modules-load.d: directory /etc/netplan: directory /etc/network: directory /etc/networkd-dispatcher: directory /etc/newt: directory /etc/opt: directory /etc/pam.d: directory /etc/perl: directory /etc/pm: directory /etc/polkit-1: directory /etc/pollinate: directory /etc/postfix: directory /etc/ppp: directory /etc/profile.d: directory /etc/python: directory /etc/python2.7: directory /etc/python3: directory /etc/python3.6: directory /etc/rc0.d: directory /etc/rc1.d: directory /etc/rc2.d: directory /etc/rc3.d: directory /etc/rc4.d: directory /etc/rc5.d: directory /etc/rc6.d: directory /etc/rcS.d: directory /etc/request-key.d: directory /etc/resolvconf: directory /etc/rsyslog.d: directory /etc/security: directory /etc/selinux: directory /etc/sensors.d: directory /etc/sgml: directory /etc/skel: directory /etc/ssh: directory /etc/ssl: directory /etc/sudoers.d: directory /etc/sysctl.d: directory /etc/systemd: directory /etc/terminfo: directory /etc/texmf: directory /etc/tmpfiles.d: directory /etc/udev: directory /etc/ufw: directory /etc/update-manager: directory /etc/update-motd.d: directory /etc/update-notifier: directory /etc/vim: directory /etc/vmware-tools: directory /etc/xdg: directory /etc/xml: directory
Magics functions
There are some magics functions that we can use in Ipython. These functions begin with a single percent symbol or a double percent symbol.
- You can obtain the full list with this command:
%lsmagic
Available line magics: %alias %alias_magic %autoawait %autocall %automagic %autosave %bookmark %cat %cd %clear %colors %conda %config %connect_info %cp %debug %dhist %dirs %doctest_mode %ed %edit %env %gui %hist %history %killbgscripts %ldir %less %lf %lk %ll %load %load_ext %loadpy %logoff %logon %logstart %logstate %logstop %ls %lsmagic %lx %macro %magic %man %matplotlib %mkdir %more %mv %notebook %page %pastebin %pdb %pdef %pdoc %pfile %pinfo %pinfo2 %pip %popd %pprint %precision %prun %psearch %psource %pushd %pwd %pycat %pylab %qtconsole %quickref %recall %rehashx %reload_ext %rep %rerun %reset %reset_selective %rm %rmdir %run %save %sc %set_env %store %sx %system %tb %time %timeit %unalias %unload_ext %who %who_ls %whos %xdel %xmode Available cell magics: %%! %%HTML %%SVG %%bash %%capture %%debug %%file %%html %%javascript %%js %%latex %%markdown %%perl %%prun %%pypy %%python %%python2 %%python3 %%ruby %%script %%sh %%svg %%sx %%system %%time %%timeit %%writefile Automagic is ON, % prefix IS NOT needed for line magics.
- With
%historywe can see the last inputs commands stored
%history
print("May the Force be with you!")
2 + 2
"geonosis".capitalize()
str.format?
print(In)
print(Out)
print(__)
!df -h
os_info = !uname -a
os_info
filetype = 'directory'
!file /etc/* | grep {filetype}
%lsmagic
%history
- %timeit or timeit calculate the time of execution.
%timeit list(range(10))
262 ns ± 24.8 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)
- There are more useful commands, that you can investigate using the
?question mark
%save?
[0;31mDocstring:[0m Save a set of lines or a macro to a given filename. Usage: %save [options] filename n1-n2 n3-n4 ... n5 .. n6 ... Options: -r: use 'raw' input. By default, the 'processed' history is used, so those magics are loaded in their transformed version to valid Python. If this option is given, the raw input as typed as the command line is used instead. -f: force overwrite. If file exists, %save will prompt for overwrite unless -f is given. -a: append to the file instead of overwriting it. This function uses the same syntax as %history for input ranges, then saves the lines to the filename you specify. It adds a '.py' extension to the file if you don't do so yourself, and it asks for confirmation before overwriting existing files. If `-r` option is used, the default extension is `.ipy`. [0;31mFile:[0m /mnt/evidence/opt/anaconda/lib/python3.7/site-packages/IPython/core/magics/code.py