shell script to download youtube video

Here is a shell script that I wrote to download youtube video (given the video URL)

This is based on youtube-dl (that's in python) tool that is available in ubuntu software repository.

I came to know that across the globe use my tool. The get in touch whenever it stops working.

How to generate and examine windows core dumps

How to generate and examine windows core dumps

• Generating core dump.
  1. install windbg(Debugging Tools for Windows). (download here : http://www.microsoft.com/whdc/devtools/debugging/installx86.Mspx)
  2. run windbg. (Start Menu->Programs->Debugging Tools for Windows->WinDbg)
  3. Attach to the process (using GUI, File->Attach to a Process... [f6])
  4. run the command

      .dump /ma c:\cpdump.dmp 

     as described here (http://wiki.zimbra.com/index.php?title=Creating_a_Core_Dump_from_a_Running_Process_using_WinDbg)
  5. This will generate the file c:\cpdump.dmp
  6. compress(zip) the file c:\cpdump.dmp and send it across

• Observing the dump file.
  1. run command

     ~*kb

     to see all the thread stacks.
  2. More information here (http://windowsitpro.com/article/articleid/21217/using-the-windbg-debugging-tool.html)

Look at help-->contents (.chm) for detailed reference

You will not be able to see symbols in the dump file if you don't have the symbol database (.pdb) file for your application. The /Z7 option that puts the symbol information in the .obj files did not help.

gotcha : when /Zi option is used and code compiled through ssh session, the com,piler fails. Running the build through windows command prompt works !


I had to compile the product with /Zi (for cl.exe) option that created vc80.pdb file.
Then during linking, use /DEBUG /PDB:/path/to/.pdb . The .pdb file generated in the link step can be used with windbg

reference:
http://msdn.microsoft.com/en-us/library/yd4f8bd1%28v=vs.71%29.aspx
http://msdn.microsoft.com/en-us/library/yd4f8bd1.aspx

you can also extract pdb from an executable that is compiled with /Z7 option (as documented here) . did not work for me though
http://support.microsoft.com/kb/258205

gratuitous ARP

what is ARP?

Arp is a protocol that is used to map ip addresses to corresponding MAC addresses. This is referred to as neighbour discovery. (ipv6 uses icmpv6 instead of ARP).
Arp is used by a machine when it wants to send an ip packet to another machine on the same LAN segment (physical LAN).

• The sender issues a 'Who has' ARP broadcast query. The machine that owns the IP address responds with the corresponding MAC address.
• The sender uses this MAC address as the destination address in the datalink packet and injects the packet on to the wire.

gratuitous ARP

gratuitous ARP is an ARP reply that is sent when there is no request. And it is a broadcast while normal ARP replies are not broadcast. This results in all machines in the segment updating their ARP cache.


Why is gratuitous ARP important

gratuitous ARP is useful to let other machines on the same subnet know any change in IP address configurations. This is particularly relevant in High Availability scenario where the active machine goes down and the standby machine takes over the new IP. If a gratuitous ARP is not sent here, the gateway(of this subnet) will continue to forward the IP packets to previously active machine.

Generally when an IP address is configured on a machine, the machine's network stack will send a gratuitous ARP.But some OSes(Like Linux) don't do that. This can be overcome on Linux by manually sending a gratuitous arp using arping command's '-A' option as follows.

arping -q -c 3 -A -I

AIX issues a gratuitous ARP when ip is configured on one of it's interfaces using ifconfig.

SQLite

I got to know about SQLite when I tried using a tool called "almanah" which is a simple diary.
"almanah" stores all the information in a file /Your/home/.local/share/diary.db.
did some search and found that this was a SQLite database file. Also figured how SQLite works. it's pretty simple.

SQLite is a set of libraries that you can link with your application. It gives a set of API's so that you can execute SQL statements as you would do with any DBMS. The difference here is that there is no other DBMS process, and everything is stored in the SQLite database file. check out this link, it's straightforward.

I was looking for a tool that would help me extract the contents of the diary into a text file.
Found this tool called "sqlite3" on linux. This is like a shell for SQLite.
run it
$ sqlite3 /Your/home/.local/share/diary.db
sqlite> .tables
entries entry_attachments entry_links
sqlite>

It shows 3 tables entries,entry_attachments and entry_links

doing a "select * from entries" gives all the entries in the diary.

the below command can be run to do it with a single command.
$sqlite3 ./diary.db "select * from entries"

Recover deleted files from deleted partition

I never thought recovering files would be so easy. I had deleted my personal files on old Laptop to return it.
I also repartitioned the disk using the Ubuntu install CD.

But I later realized later that I had deleted something that I had not backed up(so naive).

There are two Linux tools that I found useful.
1) testdisk
2) photorec

There is a project called SystemRescueCd which includes useful tools to recover data.

I used "test disk" tool to recover the deleted partition I was not able to restore the partition as such. But was able to browse files using the tool itself.

There is another tool called  "photorec" which I used to recover deleted files of that partition.

you can use the SystemRescueCd bootable cd or boot using ubuntu live cd and install the tools you want and use them as I did.

Blogged with the Flock Browser

analyze extremely large packet capture(tcpdump) file

I recently had to analyze an extremely large packet capture file to resolve a customer issue.
wireshark would crash trying to load the file(around 375 MB).
You start thinking 'why did the client not capture packets only when the problem occured?'.
But, I quickly realised that tcpdump can be used with the capture file as input and filters can be applied to extract packets of our interest.

In this case I was interested in packets that had a particular ip address. So used the below command to extracted those packets into another pcap file.

tcpdump -r [largefile.pcap] -w [filteredFile.pcap] [filter]

And now I have a pcap file that wireshark can load so that I can take a good look at what is happening.