Cisco Tips & Tricks

August 11, 2009

Tunneling Solutions in IPv6

Filed under: ccie, cisco, IPv6 — ciscotips @ 4:38 am

As usual Narbik posted an excellent post on Groupstudy on IPv6 Tunnels. Here it is

*There are 5 tunneling solution in IPv6:*


*1. Using the “Tunnel mode ipv6ip”, in this case the tunnel source and
destination are configured with IPv4 addressing and the tunnel interface is
configured with IPv6. This will use protocol 41. This is used for IPv6/IPv4.


R1(config)#int tunnel 1

R1(config-if)#ipv6 address 12:1:12::1/64

R1(config-if)#tunnel source

R1(config-if)#tunnel destination

R1(config-if)#*tunnel mode ipv6ip*


*2. Using the “Tunnel mode gre ipv6”, in this case the tunnel source and
destination are all configured with IPv6 addressing. This is used for
IPv6/IPv6. *


BB1(config)#int tunnel 1

BB1(config-if)#ipv6 address 121:1:121::111/64

BB1(config-if)#tunnel source 10:1:111::111

BB1(config-if)#tunnel destination 10:1:112::112

BB1(config-if)#*tunnel mode gre ipv6*


*3. In this case, the third type, the tunnel mode is NOT used at all, note
that the tunnel interface is configured with IPv6 and the tunnel source and
destination is configured with IPv4 but no mention of tunnel mode. This
configuration will use protocol 47. This is used for IPv6/IPv4. *


R1(config)#int tunnel 13

R1(config-if)#ipv6 address 13:1:13::1/64

R1(config-if)#tunnel source

R1(config-if)#tunnel destination


*4. Note in this case a special addressing is assigned to the tunnel
interface which is a concatenation of a reserved IPv6 address of
2002followed by the translated
IPv4 address of a given interface on the router. In this configuration ONLY
the tunnel source address is used and since the tunnel is automatic,
the destination
address is NOT configured. The tunnel mode is set to “Tunnel mode ipv6ip
6to4”. Note the IPv4 address of is translated to 0A.01.01.01 and
once concatenated, it will be “2002:0A01:0101: or 2002:A01:101. This is used
for IPv6/IPv4.*


R1(config)#interface Tunnel14

R1(config-if)#ipv6 address 2002:A01:101::/128

R1(config-if)#tunnel source

R1(config-if)#*tunnel mode ipv6ip 6to4*


*5. ISATAP, ISATAP works like 6to4 tunnels, with one major difference, it
uses a special IPv6 address which is formed as follows: *

*In this tunnel mode, the network portion can be any IPv6 address, whereas
in 6to4 it had to start with 2002.*

*Note when the IPv6 address is assigned to the tunnel interface, the
“eui-64” is used, in this case the host portion of the IPv6 address starts
with “0000.5EFE” and then the rest of the host portion is the translated
IPv4 address of the tunnel’s source IPv4 address. This translation is
performed automatically unlike 6to4. This is used for IPv6/IPv4.*


R4(config)#int tunnel 46

R4(config-if)#ipv6 address 46:1:46::/64 eui-64

R4(config-if)#tunnel source

R4(config-if)#*tunnel mode ipv6ip ISATAP*

August 7, 2009

Undocumented IOS commands

Filed under: ccie, cisco, Router — ciscotips @ 1:01 am

Undocumented IOS commands



Here you can find a collection of undocumented (not on Cisco Dokumentation CD or in online help) IOS commands:

  • csim
  • show interface switching
  • show ip ospf statistics
  • show ip ospf m
  • show ip ospf events
  • debug ip ospf monitor
  • no service password-recovery
  • ttcp
  • if-con
  • no snmp-server sparse-tables
  • who
  • show region
  • ps -c
  • quit
  • test aaa group radius
  • show snmp
  • set option flowcontrol




With the command csim you can emulate a voice call. ItŽs like sombody calls the specified number. Usefull, if you donŽt have physically access to telephone:

Sucessfull call:

wg1r1#csim start 089150
csim: called number = 089150, loop count = 1 ping count = 0
csim err csimDisconnected recvd DISC cid(21)
csim: loop = 1, failed = 1
csim: call attempted = 1, setup failed = 1, tone failed = 0


wg1r1#csim start 089151
csim: called number = 089151, loop count = 1 ping count = 0

csim err:csim_do_test Error peer not found


With show interface switching you can see in detail, which protocols are process, fast or elsewhere switched. ItŽs helpfull for traffic engineering:

wg1r1#sh interfaces switching

Throttle count          0
Drops         RP          0         SP          0
SPD Flushes   Fast        0        SSE          0
SPD Aggress   Fast        0
SPD Priority  Inputs      5700     Drops        0

Protocol       Path    Pkts In   Chars In   Pkts Out  Chars Out
Other       Process       8821     983982       7806     468360
Cache misses                 0
Fast                         0          0          0          0
Auton/SSE                    0          0          0          0
IP          Process      32228    3339712       5786     611402
Cache misses                 0
Fast                     92667    8980418     133414  189720295
Auton/SSE                    0          0          0         0
DEC MOP     Process          0          0        130      10010
Cache misses                 0
Fast                         0          0          0          0
Auton/SSE                    0          0          0          0
ARP         Process       6440     386380        195      11700
Cache misses                 0
Fast                         0          0          0          0
Auton/SSE                    0          0          0          0
CDP         Process          0          0       1302     404922
Cache misses                 0
Fast                         0          0          0          0
Auton/SSE                    0          0          0          0



With show ip ospf statistics you get detail informations about OSPF SPF. You can see the reasons for SPF and the calculation times:

London#show ip ospf statistic
Area 0: SPF algorithm executed 15 times

SPF calculation time
Delta T Intra D-Intra Summ D-Summ Ext D-Ext Total Reason
00:45:02 32 0 0 0 0 0 36 R,
00:41:00 28 0 0 0 0 0 28 R, N,
00:40:50 28 0 0 0 0 0 28 R,
00:38:13 28 0 0 0 4 0 32 R, X
00:34:18 24 0 0 0 4 0 32 R, N,SN,SA,X
00:07:37 24 0 0 0 0 0 36 R,
00:05:18 308 0 4 0 28 0 356 R,
00:04:05 24 0 0 0 4 0 28 R,
00:01:31 24 0 4 0 0 0 36 R,
00:01:21 24 0 0 4 0 0 36 R,

With debug ip ospf monitor you can monitor the OSPF SPF.

London#deb ip ospf monitor 

OSPF spf monitoring debugging is on
2d04h: OSPF: address on TokenRing0 is dead, state DOWN
2d04h: OSPF: Schedule SPF in area 0
Change in LS ID, LSA type R,
2d04h: OSPF: schedule SPF: spf_time 0ms wait_interval 187489656s
2d04h: %SYS-5-CONFIG_I: Configured from console by console
2d04h: OSPF: Start redist-scanning
2d04h: OSPF: Scan for redistribution
2d04h: OSPF: End scanning, Elapsed time 0ms
2d04h: %LINK-5-CHANGED: Interface TokenRing0, changed state to administratively down
2d04h: OSPF: Begin SPF at 0xB2CF100ms, process time 360ms
2d04h: spf_time 0ms, wait_interval 187489656s
2d04h: OSPF: End SPF at 0xB2CF120ms, Total elapsed time 32ms
2d04h: Intra:24ms, Inter: 0ms, External: 0ms


I found that in 12.0(20) Enterprise that show ip os monitor is not available but show ip os m is. Show ip os events is the undocumented command:

tr-Albany-2#sh ip os monitor
% Invalid input detected at ‘^’marker.

Rtr-Albany-2# sh ip os m
AS System 1
Maxage delete timer due in NEVER
Rtr-Albany-2#sh ip os events
1 17844 Timer Exp: if_ack_delayed 0x616DDE08
2 40152 Generic: ospf_redist_callback 0x61735760
3 59800 Timer Exp: if_ack_delayed 0x616DD3C8
4 88664 Timer Exp: if_ack_delayed 0x616DDE08
5 88672 Timer Exp: if_ack_delayed 0x616DD3C8
6 100184 Generic: ospf_redist_callback 0x61735760
7 126576 Timer Exp: if_ack_delayed 0x619621AC
8 160216 Generic: ospf_redist_callback 0x61735760
9 164976 Timer Exp: if_ack_delayed 0x616DDE08
10 189256 Timer Exp: if_ack_delayed 0x616DD3C8
90 534184 Timer Exp: nbr_retrans_lsa 0xC0A8E425
91 534184 Timer Exp: nbr_retrans_lsa 0xC0A8FC13
92 534188 Timer Exp: nbr_retrans_lsa 0xC0A8FC04


Many thanks for this command goes to Paul.


Disable Password Recovery or ROM Monitor Access:

#sh run
Building configuration…
Current configuration:
Version 12.1
no service password-recovery

Proceed with reload? [confirm]

.Feb 9 12:57:24.013: %SYS-5-RELOAD: Reload requested
System Bootstrap, Version 11.3(2)XA4, RELEASE SOFTWARE (fc1)
Copyright (c) 1999 by cisco Systems, Inc.
TAC:Home:SW:IOS:Specials for info


PC = 0xfff0bba8, Vector = 0x500, SP = 0x680127c8
C2600 platform with 49152 Kbytes of main memory


PC = 0xfff14dfc, Vector = 0x500, SP = 0x80004864
PC = 0xfff14e08, Vector = 0x500, SP = 0x80004864
PC = 0xfff14df8, Vector = 0x500, SP = 0x80004374
program load complete, entry point: 0x80008000, size: 0x8b9c04

Cisco Internetwork Operating System Software
IOS ™ C2600 Software (C2600-IS-M), Version 12.1(3a)T4, RELEASE SOFTWARE (fc1)


Start a TCP data server/receiver for TCP performance testing between two Cisco 7500 router:

sh ver
Cisco Internetwork Operating System Software
IOS ™ RSP Software (RSP-JSV-M), Version 12.0(7), RELEASE SOFTWARE (fc1)

transmit or receive [receive]: transmit
Target IP address:
perform tcp half close [n]:
send buflen [8192]:
send nbuf [2048]:
bufalign [16384]:
bufoffset [0]:
port [5001]:
sinkmode [y]:
buffering on writes [y]:
show tcp information at end [n]:

ttcp-t: buflen=8192, nbuf=2048, align=16384/0, port=5001 tcp ->
%Connect failed: Destination unreachable; gateway or host down

transmit or receive [receive]:
perform tcp half close [n]:
receive buflen [8192]:
bufalign [16384]:
bufoffset [0]:
port [5001]:
sinkmode [y]:
rcvwndsize [4128]:
delayed ACK [y]:
show tcp information at end [n]:

ttcp-r: buflen=8192, align=16384/0, port=5001
rcvwndsize=4128, delayedack=yes tcp

Many thanks for this command goes to Thomas Moyses.


On Cisco 7500 you can connect you to the VIP boards for VIP commands (memory, cpu,etc.)

Router#if-con 2 con
Entering CONSOLE for VIP2 2
Type “^C^C^C” or “if-quit” to end this session

Exec commands:
clear Reset functions
disable Turn off privileged commands
disconnect Disconnect an existing network connection
enable Turn on privileged commands
exit Exit from the EXEC
help Description of the interactive help system
lock Lock the terminal
login Log in as a particular user
logout Exit from the EXEC
name-connection Name an existing network connection
ping Send echo messages
resume Resume an active network connection
set Set system parameter (not config)
show Show running system information
systat Display information about terminal lines
terminal Set terminal line parameters
traceroute Trace route to destination
where List active connections

VIP-Slot2#sh ver
Cisco Internetwork Operating System Software
IOS ™ VIP Software (SVIP-DW-M), Version 12.0(7), RELEASE SOFTWARE (fc1)
Copyright (c) 1986-1999 by cisco Systems, Inc.
Compiled Wed 13-Oct-99 22:05 by phanguye
Image text-base: 0x60010920, data-base: 0x60224000

ROM: System Bootstrap, Version 11.1(11368) [pgreenfi 17], INTERIM SOFTWARE

VIP-Slot2 uptime is 10 minutes
System restarted by power-on
Running default software

cisco VIP2 (R4700) processor (revision 0x02) with 8192K bytes of memory.
Processor board ID 00000000
R4700 processor, Implementation 33, Revision 1.0 (512KB Level 2 Cache)
4 Ethernet/IEEE 802.3 interface(s)
1 FastEthernet/IEEE 802.3 interface(s)

Configuration register is 0x0
Exec commands:
cd Change current directory
clear Reset functions
clock Manage the system clock
configure Enter configuration mode
copy Copy from one file to another
debug Debugging functions (see also ‘undebug’)
delete Delete a file
dir List files on a filesystem
disable Turn off privileged commands
disconnect Disconnect an existing network connection
enable Turn on privileged commands
exit Exit from the EXEC
help Description of the interactive help system
lock Lock the terminal
login Log in as a particular user
logout Exit from the EXEC
more Display the contents of a file
name-connection Name an existing network connection
no Disable debugging functions
ping Send echo messages
pwd Display current working directory
reload Halt and perform a cold restart
resume Resume an active network connection
send Send a message to other tty lines
set Set system parameter (not config)
setup Run the SETUP command facility
show Show running system information
systat Display information about terminal lines
terminal Set terminal line parameters
test Test subsystems, memory, and interfaces
traceroute Trace route to destination
undebug Disable debugging functions (see also ‘debug’)
where List active connections
write Write running configuration to memory, network, or terminal


Many thanks for this command goes to Thomas Moyses.


Router(config)# no snmp-server sparse-tables 

Get the complete SNMP MIB table. On controller interface you get without this command e.g. no out bytes counter. With this commands you get every object with SNMP get-next.


who is a alias for show user.

Line User Host(s) Idle Location
* 0 con 0 idle 00:00:00
1 aux 0 idle 00:01:19

Get more detail IOS informations:

Router#sh region
Region Manager:

Start End Size(b) Class Media Name

0x02700000 0x02FFFFFF 9437184 Iomem R/W iomem
0x60000000 0x60FFFFFF 16777216 Flash R/O flash
0x80000000 0x826FFFFF 40894464 Local R/W main
0x80008088 0x8071A2A3 7414300 IText R/O main:text
0x8071A2A4 0x8097FD93 2513648 IData R/W main:data
0x8097FD94 0x80A4E0CF 844604 IBss R/W main:bss
0x80A4E0D0 0x826FFFFF 30089008 Local R/W main:heap

Many thanks for this command goes to Thomas Moyses.


router> ps -c

Will list the process on the router in a “better” way then show process.

Many thanks for this command goes to Thomas Surber.


leaving a telnet session: normally with “exit” or “ex”.
do this very fast with “q” (quit)

Many thanks for this command goes to Peter Schmid.


test aaa group radius toto titi

To test radius on one router.

CISCO AS5800: This command allow to connect directly to the DSC IOS card from the router Shelf of the

dsip con slave “Slot # of the DSC”
ex: dsip con slave 12

Many thanks for this commands goes to Serge Baikoff.


These commands have been found in a Cisco 3640 router using the version 12.1(6). It’s very posible that some of these commands are not avalaible in older versions.

— show ip ftp-username

Show the name of the user configured with the comand “ip ftp username”.

–show ip ftp-password

Display the password configured with the command “ip ftp passwword”

— show ip spd

I don’t know the function of this command, but it looks as folow:

sh ip spd
Current mode: normal.
Queue min/max thresholds: 73/74, Headroom: 100
IP normal queue: 0, priority queue: 0.
SPD special drop mode: none

— show ip ospf timers lsa

Show the ospf lsa timers and its output is:

sh ip ospf timers lsa

OSPF Router with ID ( (Process ID 1)

Group size 5, Head 0, Search Index 4, Interval 240 sec
Next update due in 00:00:15
Current time 64496
Index 0 Timestamp 64511
Index 1 Timestamp 64768
Index 2 Timestamp 65014
Index 3 Timestamp 65258
Index 4 Timestamp 65509

Failure Head 0, Last 0 LSA group failure logged

— show ip eigrp sia-statistics

I don’t know the meaning of this command. When it’s typed shows no useful information:

sh ip eigrp sia-statistic

Many thanks for this command goes to Atreides.


Here’s some additional hidden cisco IOS command for debugging ATM on a PA-A3. You need to connect to the VIP first and be in enable mode: (I guess this one is double hidden, first the if-con and then the test atmdx)

syntax: test atmdx PAslotnr (0 or 1)

7507#if-con 0
Console or Debug [C]: c
Entering CONSOLE for VIP2 R5K 0
Type “^C^C^C” or “if-quit” to end this session

VIP-Slot0#test atmdx 0

1 – VC test
2 – Rx SAR
3 – Tx SAR
q – Quit
Select option : 1

Test VC Section:
1 – Show vc
2 – Show vc statistics
3 – Change vc
q – Quit
Select option : 2
Enter vc number or -1 for all : -1
vcd 1: in_pkts 4, in_drops 0, out_pkts 4, out_drops 0
crc32 0, timeout 0, oversized 0
vcd 2: in_pkts 1, in_drops 0, out_pkts 1, out_drops 0
crc32 0, timeout 0, oversized 0
vcd 3: in_pkts 28, in_drops 0, out_pkts 21, out_drops 0
crc32 0, timeout 0, oversized 0
vcd 4: in_pkts 1, in_drops 0, out_pkts 1, out_drops 0
crc32 0, timeout 0, oversized 0

Many thanks for this commands goes to Rogger Schobben.


Tested on 12.22T:

sh snmp host
sh snmp notif
test aaa group policy accounting WORD
test aaa group policy authorization WORD
test aaa group policy authentication WORD
test aaa group priviliges authentication WORD (same for accounting and auth)

In fact there’s a bunch of other stuff you can do behind test aaa group. If you type “a” behind this f.ex., you will also see incomplete command.

Many thanks for this commands goes to Filip Waeytens.


If you are overrunning the buffers on ports, at a Cat6500 enable (global) mode, we can enable flow control between the two ASICs on the 6348 card (HW MSFC2) using the command

set option flowcontrol enable

This should help with the out-discards. It may clear it up and it may not. If it doesn’t, you will need more to either create an etherchannel to the router or move to a higher bandwidth connection. The command set option flowcontrol enable allows the COIL ASIC to pass traffic to the Pinnacle ASIC for buffering assistance when the COIL ASIC is overloaded with traffic.

Many thanks for this commands goes to Renato Vilela de Magalháes.


bgp common-administration

bgp dynamic-med-interval

bgp process-dpa

clear ip eigrp [as] event Clear IP-EIGRP event logs

clear ip eigrp [as] logging Stop IP-EIGRP event logging

config overwrite

debug dialer detailed

debug ip packet … dump Outputs a hex & ASCII dump of the packet’s contents

debug isdn code

debug sanity

if-con attach to a vip console


ip forwarding

ip forwarding accounting

ip forwarding accounting adjacency-update

ip forwarding accounting non-recursive

ip forwarding accounting per-prefix

ip forwarding accounting prefix-length

ip forwarding switch

ip forwarding traffic-statistics

ip forwarding traffic-statistics load-interval

ip forwarding traffic-statistics update-rate

ip igmp

ip igmp immediate-leave

ip igmp immediate-leave group-list

ip local-pool

ip ospf-name-lookup

ip slow-converge

ip spd

ip spd mode

ip spd mode aggressive

ip spd queue

ip spd queue max-threshold

ip spd queue min-threshold

memory scan Parity check for 7500 RSPs

modem-mgmt csm debug-rbs

no service password-recovery

service internal

set destination-preference

show alignment

show asp

show caller

show caller ip

show chunk

show chunk summary

show controller vip log

show controller vip tech

show fib

show fib drop

show fib interface

show fib interface detail

show fib interface loopback

show fib interface null

show fib interface statistics

show fib interface vlan

show fib linecard

show fib linecard detail

show fib not-cef-switched

show fib not-fib-switched

show hardware

show idb

show interface statis

show interface switching

show interfaces stat

show interface stat

show interfaces switching

show int switching

show ip eigrp event [as] [start# end#] IP-EIGRP Events

show ip eigrp sia-event [as] [start# end#] IP-EIGRP SIA event

show ip eigrp timers [as] IP-EIGRP Timers

show ip ospf bad-checksum

show ip ospf delete

show ip ospf delete-list

show ip ospf ev

show ip ospf events

show ip ospf maxage

show ip ospf maxage

show ip ospf statistics

show isdn active

show isdn history

show list

show list nonempty

show llc

show media

show media access-lists

show modem mapping

show parity

show parser

show parser links

show parser modes

show parser unresolved

show profile

show profile detail

show profile terse

show refuse-message

show region

show region address

show rsh

show rsh-disable-commands

show rsp

show slip

show slot

show snmp mib

show sum

show timers

snmp-server priority low

test crash makes the router crash

test ipc misc

test mbus power




router bgp …

bgp redistribute-internal


Redistributing BGP into another protocol only redistributes E-BGP routes. Using this command in the BGP configuration will also redistribute I-BGP routes in the other routing-protocol.



August 5, 2009

Protocol Overhead

Filed under: cisco, Technology and Software — ciscotips @ 9:40 pm

Protocol Overhead

How fast can you really go using a given media and protocol stack? We examine how much bandwidth is left for applications.


Ethernet frame format:

  • 6 byte dest addr
  • 6 byte src addr
  • [4 byte optional 802.1q VLAN Tag]
  • 2 byte length/type
  • 46-1500 byte data (payload)
  • 4 byte CRC
Ethernet overhead bytes:
  12 gap + 8 preamble + 14 header + 4 trailer = 38 bytes/packet w/o 802.1q
  12 gap + 8 preamble + 18 header + 4 trailer = 42 bytes/packet with 802.1q

Ethernet Payload data rates are thus:
  1500/(38+1500) = 97.5293 %   w/o 802.1q tags
  1500/(42+1500) = 97.2763 %   with 802.1q tags

TCP over Ethernet:
 Assuming no header compression (e.g. not PPP)
 Add 20 IPv4 header or 40 IPv6 header (no options)
 Add 20 TCP header
 Add 12 bytes optional TCP timestamps
 Max TCP Payload data rates over ethernet are thus:
  (1500-40)/(38+1500) = 94.9285 %  IPv4, minimal headers
  (1500-52)/(38+1500) = 94.1482 %  IPv4, TCP timestamps
  (1500-52)/(42+1500) = 93.9040 %  802.1q, IPv4, TCP timestamps
  (1500-60)/(38+1500) = 93.6281 %  IPv6, minimal headers
  (1500-72)/(38+1500) = 92.8479 %  IPv6, TCP timestamps
  (1500-72)/(42+1500) = 92.6070 %  802.1q, IPv6, ICP timestamps

UDP over Ethernet:
 Add 20 IPv4 header or 40 IPv6 header (no options)
 Add 8 UDP header
 Max UDP Payload data rates over ethernet are thus:
  (1500-28)/(38+1500) = 95.7087 %  IPv4
  (1500-28)/(42+1500) = 95.4604 %  802.1q, IPv4
  (1500-48)/(38+1500) = 94.4083 %  IPv6
  (1500-48)/(42+1500) = 94.1634 %  802.1q, IPv6

An excellent source of ethernet information is Charles Spurgeon’s Ethernet Web Site.


  1. 48-bit (6 byte) ethernet address have a 24-bit “Organizationally Unique Identifier” (OUI) assigned by IEEE + a 24-bit number assigned by the vendor.
  2. The minimum ethernet payload (data field) is 46 bytes which makes a 64 byte ethernet packet including header and CRC.
  3. The maximum ethernet payload (data field) is 1500 bytes which makes a 1518 byte ethernet packet including header and CRC. When 802.1q added an optional 4-byte VLAN Tag Header, they extended the allowed maximum frame size to 1522 bytes (22 byte header+CRC).
  4. The bit speed of 100 Mbps ethernet on the wire/fiber is actually 125 Mbps due to 4B/5B encoding. Every four data bits gets mapped to one of 16 5-bit symbols. This leaves 16 non-data symbols. This encoding came from FDDI.
  5. The original Ethernet II spec had a two byte type field which 802.3 changed to a length field, and later a length/type field depending on use: values 1536 and over are types, under 1536 lengths.

Gigabit Ethernet with Jumbo Frames

Gigabit ethernet is exactly 10 times faster than 100 Mbps ethernet, so for standard 1500 byte frames, the numbers above all apply, multiplied by 10. Many GigE devices however allow “jumbo frames” larger than 1500 bytes. The most common figure being 9000 bytes. For 9000 byte jumbo frames, potential GigE throughput becomes (from Bill Fink, the author of nuttcp):

Theoretical maximum TCP throughput on GigE using jumbo frames:

	(9000-20-20-12)/(9000+14+4+7+1+12)*1000000000/1000000 = 990.042 Mbps
	  |   |  |  |     |   |  | | | |       |         |
	 MTU  |  |  |    MTU  |  | | | |      GigE      Mbps
	      |  |  |         |  | | | |
	     IP  |  |  Ethernet  | | | |      InterFrame Gap (IFG), aka
	  Header |  |    Header  | | | |      InterPacket Gap (IPG), is
		 |  |            | | | |      a minimum of 96 bit times
	       TCP  |          FCS | | |      from the last bit of the
	    Header  |              | | |      FCS to the first bit of
		    |       Preamble | |      the preamble
		  TCP                | |
	      Options            Start |
	  (Timestamp)            Frame |
			     Delimiter |
				 (SFD) |

Theoretical maximum UDP throughput on GigE using jumbo frames:

	(9000-20-8)/(9000+14+4+7+1+12)*1000000000/1000000 = 992.697 Mbps

Theoretical maximum TCP throughput on GigE without using jumbo frames:

	(1500-20-20-12)/(1500+14+4+7+1+12)*1000000000/1000000 = 941.482 Mbps

Theoretical maximum UDP throughput on GigE without using jumbo frames:

	(1500-20-8)/(1500+14+4+7+1+12)*1000000000/1000000 = 957.087 Mbps


An excellent paper on ATM overhead was written by John Cavanaugh of MSC. A postscript copy can be found here. Based on that paper:

  -------------------------- DS3 ------------------------------
  Line Rate           44.736 Mbps
  PLCP Payload        40.704                       (avail to ATM)
  ATM Payload         36.864                       (avail to AAL)
                     MTU=576  MTU=9180 MTU=65527
  AAL5 Payload        34.501   36.752   36.845     (avail to LLC/SNAP)
  LLC/SNAP Payload    34.028   36.720   36.841     (avail to IP)
  IP Payload          32.847   36.640   36.830     (avail to transport)
    UDP Payload       32.374   36.608   36.825     (avail to application)
    TCP Payload       31.665   36.560   36.818     (avail to application)

  -------------------------- OC-3c ------------------------------
  Line Rate           155.520 Mbps
  SONET Payload       149.760                      (avail to ATM)
  ATM Payload         135.632                      (avail to AAL)
                     MTU=576  MTU=9180 MTU=65527
  AAL5 Payload        126.937  135.220  135.563    (avail to LLC/SNAP)
  LLC/SNAP Payload    125.198  135.102  135.547    (avail to IP)
  IP Payload          120.851  134.808  135.506    (avail to transport)
    UDP Payload       119.112  134.690  135.489    (avail to application)
    TCP Payload       116.504  134.513  135.464    (avail to application)

  -------------------------- OC-12c -----------------------------
  Line Rate           622.080 Mbps
  SONET Payload       600.768                      (avail to ATM)
  ATM Payload         544.092                      (avail to AAL)
                     MTU=576  MTU=9180 MTU=65527
  AAL5 Payload        509.214  542.439  543.818    (avail to LLC/SNAP)
  LLC/SNAP Payload    502.239  541.966  543.752    (avail to IP)
  IP Payload          484.800  540.786  543.586    (avail to transport)
    UDP Payload       477.824  540.313  543.519    (avail to application)
    TCP Payload       467.361  539.605  543.420    (avail to application)


  1. DS3 and SONET frames are 125 usec long (8000/sec).
  2. PLCP packs 12 ATM cells per DS3 frame, for 96 kc/s (8000×12).
  3. An STS-3c frame (OC3c) is 2430 bytes long (270 bytes x 9 rows), 90 of which are consumed by SONET overhead (9 bytes x 9 rows section and line overhead and 1 byte x 9 rows path overhead), 2340 bytes are payload (260 bytes x 9 rows). The payload is called the Synchronous Payload Envelope (SPE).
  4. An STS-12c frame (OC12c) is 9720 bytes long, 333 of which are SONET overhead, 9387 bytes are payload (SPE). Note that this is slightly larger than four STS-3c SPE’s (4×2340=9360), the advantage of “concatenated” OC12c vs. OC12.
  5. ATM cells are 53 bytes long: 5 header and 48 payload.
  6. AAL5 adds an 8 byte trailer in the last 8 bytes of the last cell, padding in front of the trailer if necessary. This results in 0-47 bytes of padding in an AAL5 frame. In the worse case, you have seven bytes of padding in one cell, and 40 bytes of padding plus the 8 byte AAL5 trailer in the following cell.
  7. RFC1483 defines two types of protocol encapsulation in AAL5
    • LLC/SNAP – adds an 8 byte header containing LLC (3 bytes), OUI (3 bytes), and PID/EtherType (2 bytes)
    • VC-mux – adds no additional bytes by sending only a single protocol type per VC
  8. IPv4 usually adds 20 bytes. IPv6 would add 40 bytes. Plus any options but assumed zero here.
  9. UDP adds an 8 byte header. (ICMP is also an 8 byte header)
  10. TCP adds a 20 byte header plus any options. A common option on high performance flows is timestamps which consume an additional 12 bytes per packet.

On the physical layer (single pt-to-pt hop), one out of every 27 cells is an OAM cell. The above calculations don’t take that into account, but that’s another 3.7% reduction!

We should add calculations for ping packets and 1500 byte packets.

So what is the largest packet that we can fit in a single ATM cell? If you are using AAL5, you have a 40 byte payload to work with. For IPv4, you could have a 20 byte header + a 20 byte IP payload. A UDP or ICMP payload could be up to 12 bytes (both use 8 bytes after the IP header). So a “ping -s8” through “ping -s12” should fit in one ATM cell and still give you a round trip time.


Packet Over SONET (POS)

Packet over SONET (POS) uses PPP with HDLC to frame IP packets. These add a five byte header and a four byte trailer under normal circumstances. No padding is required, except for any possible idle time between packets. Byte stuffing is used (see notes below) which can expand the length of the POS frame.

       Flag Byte (0x7e)
       Address Byte (0xff = all stations)
       Control Byte (0x03 = Unnumbered Information)
          Protocol - 2 bytes, 1 byte if compressed      +
          Payload - 0-MRU bytes                         | PPP part
          Padding - 0+ bytes                            +
       Frame Check Sequence (FCS) - 4 bytes (2 in limited cases)
       Flag Byte (0x7e)
       [Interframe fill or next Address]

HDLC has no set frame size limit, nor does PPP specify the payload size, you just keep reading until you see a Flag byte. PPP however specifies that the Maximum Receive Unit (MRU) default is 1500 bytes and that other sizes can be negotiated using LCP. These LCP messages have a 16-bit length field, so a properly negotiated maximum payload would be 65535 bytes. [It would be possible to configure a sender/receiver pair to go beyond 65535 and simply not negotiate a size with LCP. No one does this however.] Most POS hardware seems to have a 4470 or 9180 byte MRU.

So we get:

  -------------------------- OC-3c ------------------------------
  Line Rate           155.520 Mbps
  SONET Payload       149.760                      (avail to POS)
  POS Payload         *** to do ***                (avail to IP)

  -------------------------- OC-12c -----------------------------
  Line Rate               622.080 Mbps
  SONET Payload           600.768                      (avail to POS)
                         MTU=1500   MTU=9000
  POS Payload (no stuff)  597.185    600.168           (avail to IP)  9 overhead
  POS Payload (rnd stuff) 592.583    595.520                          20.71875 overhead
  POS Payload (max stuff) 299.486    300.234                          1509 overhead

  ~TCP Payload w/ts rnd   572.040    592.079


  1. Only one flag byte is required between frames, i.e. the flag byte that ends one frame can also begin the next.
  2. It is possible for the HDLC Address and Control fields to be “compressed”, i.e. non-existent. This is negotiated by PPP’s Link Control Protocol (LCP). The RFC’s however recommend that they be present on high speed links and POS.
  3. The protocol field can be compressed to one byte (negotiated by LCP), but this is also discouraged on high speed links and POS.
  4. IP -> PPP -> FCS generation -> Byte stuffing -> Scrambling -> SONET/SDH framing
  5. The Frame Check Sequence (FCS) for POS should be 32-bits. RFC2615 allows for 16-bits (the PPP default) only when required for backward compatibility, and only on OC3c. Even on OC3c 32-bit is recommended. The FCS length is configured, not negotiated. The FCS-32 uses the exponents x**0, 1, 2, 4, 5, 7, 8, 10, 11, 12, 16, 22, 23, 26, 32.
  6. Byte stuffing escapes any Flag (0x7e) and Escape (0x7d) bytes by inserting an Escape byte and xoring the original byte with 0x20. [PPP can also escape negotiated control characters but this is not used in POS.] Byte stuffing can at worse double the payload size (e.g. data of all 0x7e). For uniform random data one in every 128 bytes would be stuffed, for an overhead of 0.775%.
  7. The stuffed data is then scrambled with 1+x**43 (the same used for ATM) to prevent certain data patterns from interfering with SONET.



  • RFC1661 The Point-to-Point Protocol (PPP), July 1994
  • RFC1662 PPP in HDLC-like Framing, July 1994
  • RFC2615 PPP over SONET/SDH, June 1999


POS with Frame Relay encapsulation

Frame Relay (FR) encapsulation can be used on POS instead of HDLC/PPP. There are not any RFC’s about Frame Relay over SONET, nor does the Multiprotocol over Frame Relay RFC1490 discuss SONET or POS, but Cisco starting doing this and others have followed.


  • RFC2427 Multiprotocol Interconnect over Frame Relay, September 1998

Generic Framing Proceedure

A new way to do POS uses PPP over GFP-F (Generic Framing Proceedure, Framed) instead of HDLC. In both the HDLC and GFP-F cases, SONET / SDH VCAT (Virtual Concatenation) is used. GFP-F also allows Ethernet frames (100, GE and 10GE) and Resilient Packet Ring (RPR) frames to be sent over SONET/SDH VCAT. GFP can also map to G.709 (part of the Optical Transport Network (OTN) series).

A GFP User Frame:

  • 4 byte Core Header
    • 2 byte PDU Length Indicator (PLI)
    • 2 byte Core Header Error Control (cHEC)
  • Payload – up to 65535 bytes
    • Payload Header (4-64 bytes)
      • 2 byte Type
      • 2 byte tHEC
      • 0-60 byte Extension Header including an optional 2 byte eHEC at the end.
    • Payload (min of 1600 should be supported, larger by agreement)
    • Payload FCS (optional)

A PLI of 0-3 indicates a GFP control frame. cHEC is a CRC-16 that protects the core header only (single bit error correction, multi bit error detection).


Multi Protocol Label Switching (MPLS)

Multi-Protocol Label Switching (MPLS) adds four bytes to every frame. As described in RFC3032 the 32-bit label includes:

 0                   1                   2                   3
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Label
|                Label                  | Exp |S|       TTL     | Stack
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Entry

     Label:  Label Value, 20 bits
     Exp:    Experimental Use, 3 bits
     S:      Bottom of Stack, 1 bit
     TTL:    Time to Live, 8 bits

Serial Lines (T1,T3)

To do

  • DS-3 is specified as 44.736 Mbps +/- 20 parts per million (ppm). So one DS-3 can vary from another by up to 1789 bps.
  • Bit-stuffing is used to accommodate rate mismatches as you mux up the DS-n hierarchy.

P. Dykstra,, March 2001, last update April 2003



Cisco says there’s no CCIE amnesty program

Filed under: ccie, cisco — ciscotips @ 4:00 pm

I found the following post on Network World By Brad Resse.

It looks like that Brad did his homework and asked Cisco on Amnesty program. 

Brad Resse  says:-

Last month I noticed that a CCIE amnesty program was being promoted all over the Internet:

Google search results for:

CCIE amnesty program

Scanning the Google search results, I found no direct links to Cisco that would provide further information about the CCIE amnesty program.

So I contacted Cisco directly to learn more about the program and received the following official Cisco response:

“Hi Brad, there is no CCIE Amnesty program in place. The post referred to is not representative of Cisco policy.”


August 3, 2009

MPLS, VRF-lite in CCIEv4

Filed under: bgp, ccie, cisco, MPLS, VPNv4, VRF, VRF-lite — ciscotips @ 1:50 pm

Now that I am preparing for version 4 . I started reading MPLS configuration on IOS software by Cisco press. I am not very comfortable with just reading book and doing nothing. I chose this book because it has some mini scenarios which I can simulate on my dynamips home lab.

I liked the book but then I remembered seeing a post by Antonio on group study that he created his own mini scenarios for studying CCIE SP. Although CCIE SP goes pretty beyond on what we need for R&S but I thought it would be a good practice to try some mini scenarios which are relevant for CCIE R&S. I am sharing the links to mini Scenarios and videos made by Antonio.

Thanks to Antonio for his amazing MPLS mini labs and videos.

Blog at