Linux-2.6.25 TCPIP函數調用大致流程
學習目的，隨手筆記。函數和文字說明會不斷補充更新。

Changelog
2008.10.08??? 最近找工作忙。暫時緩緩

插口層
系統調用
send
??? sys_send
??????? sys_sendto
sendto
??? sys_sendto
??????? sock_sendmsg
sendmsg
??? sys_sendmsg
??????? sock_sendmsg
write
??? sys_write
??????? vfs_write
??????????? file->f_op->write = do_sync_write
??????????????? filp->f_op->aio_write = sock_aio_write
??????????????????? do_sock_write
??????????????????????? __sock_sendmsg
writev
??? sys_writev
??????? vfs_writev
??????????? do_readv_writev
??????????????? do_sync_readv_writev
??????????????????? sock_aio_write
??????????????????????? do_sock_write
??????????????????????????? __sock_sendmsg
recv
??? sys_recv
??????? sys_recvfrom
recvfrom
??? sys_recvfrom
??????? sock_recvmsg
recvmsg
??? sys_recvmsg
??????? sock_recvmsg
read
??? sys_read
??????? vfs_read
??????????? file->f_op->read= do_sync_read
??????????????? filp->f_op->aio_read= sock_aio_read
??????????????????? do_sock_read
??????????????????????? __sock_recvmsg
readv
??? sys_readv
??????? vfs_readv
??????????? do_readv_readv
??????????????? do_sync_readv_readv
??????????????????? sock_aio_read
??????????????????????? do_sock_read
??????????????????????????? __sock_recvmsg
socket
listen
connect
bind
select
close
shutdown
ioctl
getsockname
getpeername
setsockopt
getsockopt

內部實現函數
sock_sendmsg
??? __sock_sendmsg

__sock_sendmsg
??? sock->ops->sendmsg
對于TCP就是tcp_sendmsg，否則就是inet_sendmsg。后者調用sk->sk_prot->sendmsg，會繼續分用為udp_sendmsg或raw_sendmsg函數

sock_recvmsg
??? __sock_recvmsg

__sock_recvmsg
??? sock->ops->recvmsg = sock_common_recvmsg
??? sock_common_recvmsg對于不同協議，是tcp_recvmsg，udp_sendmsg或raw_sendmsg函數。

運輸層

TCP
系統調用sys_connect間接調用了tcp_v4_connect
tcp_v4_connect
??? ip_route_connect(尋找路由)
??? __ip_route_output_key
??? ip_route_output_flow★
??? tcp_connect(構造一個SYN并發送)
??????? tcp_transmit_skb
??????? inet_csk_reset_xmit_timer(啟動一個超時定時器,等待SYN+ACK)


TCP的寫函數最終都調用了tcp_sendmsg
tcp_sendmsg★
??? __tcp_push_appending_frames
??????? tcp_write_xmit
??????????? tcp_transmit_skb
??? tcp_push_one
??????? tcp_transmit_skb???
??? tcp_push
??????? __tcp_push_pending_frames
???????
TCP發送數據共有三種途徑__tcp_push_appending_frames,tcp_push_one,tcp_push，其中tcp_push調用了__tcp_push_pending_frames。到底調用哪個或哪些函數取決于是否有PUSH標志、NAGLE是否開啟、和一些其他情況。
__tcp_push_appending_frames是試圖一次發送完緩存隊列中所有的skb。tcp_push_one先計算擁塞窗口，然后只發送窗口大小的數據，如果窗口大小為0，則不發送任何數據。


TCP實際的發送函數，tcp_transmit_skb
/* This routine actually transmits TCP packets queued in by
* tcp_do_sendmsg().? This is used by both the initial
* transmission and possible later retransmissions.
* All SKB's seen here are completely headerless.? It is our
* job to build the TCP header, and pass the packet down to
* IP so it can do the same plus pass the packet off to the
* device.
*
* We are working here with either a clone of the original
* SKB, or a fresh unique copy made by the retransmit engine.
*/
tcp_transmit_skb
??? build包頭
??? icsk->icsk_af_ops->queue_xmit = ip_queue_xmit★


硬件->IP層->運輸層收到數據，添加到對應的SOCKET緩沖區中，回復ACK
由ip_rcv間接調用
tcp_v4_rcv
??? __inet_lookup(根據一些參數，查找sock結構)
??????? __inet_lookup_established(在已經建立的連接中找，通過inet_lhashfn在哈希表中查找)
??????? __inet_lookup_listener(在監聽中的Socket中找，通過inet_lhashfn在哈希表中查找)
??? tcp_v4_do_rcv
??????? tcp_rcv_established(ESTABLISHED)★
??????? tcp_child_process
??????????? tcp_rcv_state_process
??????? tcp_rcv_state_process(除ESTABLISHED和TIME_WAIT之外)★
??? tcp_prequeue(見后面詳細解釋)
??????? sk->sk_backlog_rcv = tcp_v4_do_rcv(又回到開頭)
??? sk_add_backlog(見后面詳細解釋)
??? tcp_timewait_state_process(TIME_WAIT)
??? tcp_v4_timewait_ack(TIME_WAIT)
??????? tcp_v4_send_ack(發送ACK)

sock結構被初始化的時候，發送和接收數據的緩沖隊列也被初始化完成，接收數據用到以下三個隊列:
sk->receive_queue
sk->prequeue
sk->sk_backlog
sk->prequeue:如果sk沒有被用戶態程序鎖定,則先進入prequeue
sk->receive_queue:接收到數據包的sk_buff鏈表隊列，如果數據包過多，造成receive_queue滿，或者sock被用戶程序鎖定，將轉入sk_backlog
sk->sk_backlog:當sock_owned_by_user函數返回真時候，(sk)->sk_lock.owner被鎖定，使用sk_add_backlog()函數(該函數實現非常簡單，只是一個為鏈表添加節點的動作)將SKB加入這個后備隊列。


tcp_rcv_established
TCP接受里面最主要的就是tcp_rcv_established和tcp_rcv_state_process了
tcp_rcv_established★
??? if(fast path)
??????? 檢查包頭各字段
??????? tcp_ack(處理CK)
??????? tcp_data_snd_check(發送ACK)
??????? __skb_pull(騰出空間)
??????? __skb_queue_tail(把數據追加到接受緩沖區)
??? else(slow path)
??????? tcp_data_queue
??????????? 對滑動窗口、序號做出處理
??????????? __skb_pull
??????????? __skb_queue_tail
??????????? tcp_event_data_recv(更新狀態)


tcp_rcv_state_process
TCP協議的狀態機，狀態轉移函數。ESTABLISHED和TIME_WAIT狀態之外的其他狀態都會調用此函數
tcp_rcv_state_process★
??? icsk->icsk_af_ops->conn_request(是tcp_v4_conn_request，LISTEN狀態)
??????? tcp_v4_send_synack(發送SYN+ACK)
??????????? ip_build_and_send_pkt
??????????????? ip_local_out
??????????????????? __ip_local_out
??????????????????????? nf_hook(dst_output)
??????????????????? dst_output
??? tcp_rcv_synsent_state_process(SYN_SENT)
??? tcp_reset
??? tcp_ack(收到ACK)
??? tcp_set_state(SYN_RECV->ESTABLISHED或者FIN_WAIT1->FIN_WAIT2)
??? tcp_time_wait(CLOSING->TIME_WAIT)
??? tcp_update_metrics(LAST_ACK)
??? ...(都是和TCP協議狀態轉移相關的東西，這里目的是打通上下，以后慢慢分析)


還有兩個出鏡率較高的函數tcp_v4_send_reset和tcp_v4_send_ack
tcp_v4_send_reset(發送RST)
??? ip_send_reply
??????? ip_route_output_key
??????? ip_push_pending_frames


tcp_v4_send_ack(發送ACK)
??? ip_send_reply
??????? ip_route_output_key
??????? ip_push_pending_frames


用戶子上而下的讀函數都間接的調用了tcp_recvmsg
tcp_recvmsg★
skb_copy_datagram_iovec
tcp_recv_urg(接受一個字節的URG數據)

UDP
UDP的寫函數都調用了udp_sendmsg
udp_sendmsg★
??? ip_route_output_flow
??? ip_append_data
??? udp_flush_pending_frames
??????? ip_flush_pending_frames
??? udp_push_pending_frames
??????? ip_push_pending_frames

硬件->IP層->運輸層收到數據，添加到對應的SOCKET緩沖區中
由ip_rcv間接調用
udp_rcv
??? __udp4_lib_rcv
??????? if(是多播或廣播)
??????????? __udp4_lib_mcast_deliver
??????????????? udp_queue_rcv_skb(對每個需要接受的UDP SOCKET緩沖調用)
??????? __udp4_lib_lookup
??????? udp_queue_rcv_skb


把數據塊sk_buff放到一個sock結構的接受緩存的末尾中
udp_queue_rcv_skb
??? sock_queue_rcv_skb
??????? skb_queue_tail

用戶子上而下的讀函數都間接的調用了udp_recvmsg
udp_recvmsg★
__skb_recv_datagram
skb_copy_datagram_iovec
skb_copy_and_csum_datagram_iovec

原始套接字
RAW Socket的寫函數都調用了raw_sendmsg
raw_sendmsg★
??? ip_route_output_flow
??? if(設置了IP_HDRINCL選項，即自己構造ip頭部)
??????? raw_send_hdrinc★
??? else
??????? ip_append_data
??????? ip_flush_pending_frames或
??????? ip_push_pending_frames


自底向上的收包
raw_rcv
由ip_forward調用ip_call_ra_chain，然后再調用的raw_rcv
raw_rcv
sock_queue_rcv_skb
skb_queue_tail
sk->sk_data_ready = sock_def_readable
waitqueue_active
sk_wake_async


用戶子上而下的讀函數都間接的調用了raw_recvmsg
raw_recvmsg★
skb_recv_datagram
__skb_recv_datagram
wait_for_packet(如果沒有數據，則調用此函數等待數據)


ICMP
在任何需要發送ICMP報文的時候都會調用此函數
icmp_send
??? __ip_route_output_key
??????? ip_route_output_slow
??? ip_route_output_key
??????? ip_route_output_flow
??? icmp_push_reply???
??????? ip_append_data
??????? ip_flush_pending_frames或
??????? ip_push_pending_frames


硬件->IP層->運輸層收到ICMP數據，作出處理邏輯
由ip_rcv間接調用
icmp_rcv
??? 完全就是icmp協議的處理邏輯，通過函數指針icmp_pointers[icmph->type].handler調用了一下函數中的某一個
??? icmp_discard
??? icmp_unreach
??? icmp_redirect
??? icmp_timestamp
??? icmp_address
??? icmp_address_reply
??? icmp_echo


網絡層
IP發送
網絡層中主要的發送函數有以下三個：ip_push_pending_frames，ip_queue_xmit，raw_send_hdrinc
ip_push_pending_frames★
將所有pending狀態的IP分組組合成一個IP分組，并發送
??? ip_local_out


ip_queue_xmit★
??? ip_route_output_flow(找路由)
??? ip_local_out


raw_send_hdrinc★
??? NF_HOOK(dst_output)


ip_local_out★
??? __ip_local_out
??????? nf_hook(dst_output)
??? dst_output

路由選擇
ip_route_output_flow★
??? __ip_route_output_key
??????? ip_route_output_slow

路由選擇
ip_route_output_slow★
??? fib_lookup
??? ip_mkroute_output
??????? __mkroute_output
??????? rt_hash
??????? rt_intern_hash
??????????? arp_bind_neighbour
??????????????? __neigh_lookup_errno
??????????????????? neigh_lookup
??????????????????? neigh_create


dst_output★
??? dst->output = ip_output
??? NF_HOOK_COND(ip_finish_output)
??????? dst_output
??????? ip_fragment
??????? ip_finish_output2
??????????? neigh_hh_output
??????????????? hh->hh_output = dev_queue_xmit★
??????????? dst->neighbour->output = neigh_resolve_output
??????????????? neigh->ops->queue_xmit = dev_queue_xmit★

IP接受
接收IPv4包，由netif_rx間接調用
ip_rcv★
??? NF_HOOK
??? ip_rcv_finish
??????? ip_route_input
??????? dst_input
??????????? dst->input(可能是ip_local_deliver或ip_forward)
??????????? if(是發給本地的包)
??????????????? dst->input是ip_local_deliver
??????????????????? NF_HOOK
??????????????????? ip_local_deliver_finish
??????????????????? ipprot->handler(可能是tcp_v4_rcv,udp_rcv,icmp_rcv,igmp_rcv)
??????????? else
??????????????? dst->input是ip_forward

更新路由
ip_route_input★
??? ip_route_input_mc(多播)
??????? rt_hash
??????? rt_intern_hash
??? ip_route_input_slow(其它)
??????? ip_mkroute_input
??????????? __mkroute_input
??????????? rt_hash
??????????? rt_intern_hash
每收到一個IP報文都會調用此函數更新路由表。ip_route_input函數的上半部分是在hash table尋找路由項，如果找到就返回。找不到才會調用后面的ip_route_input_mc或ip_route_input_slow來更新路由表。


轉發
ip_forward★
ip_call_ra_chain
raw_rcv★
??? xfrm4_route_forward(處理路由)
??????? xfrm_route_forward
??????????? __xfrm_route_forward
??????????????? xfrm_lookup
??????????????????? __xfrm_lookup
??????????????????????? xfrm_find_bundle
??????????????????????????? afinfo->find_bundle = __xfrm4_find_bundle
??????????????????????? xfrm_bundle_create
??????????????????????????? xfrm_dst_lookup
??????????????????????????????? afinfo->dst_lookup = xfrm4_dst_lookup
??????????????????????????????????? __ip_route_output_key
??????????????????????????????????????? ip_route_output_slow★
??? 處理各個參數(在一定條件下發送ICMP)
??? ip_decrease_ttl(減少TTL)
??? NF_HOOK(ip_forward_finish)
??????? dst_output


鏈路層
接收幀
由硬件驅動在中斷處理程序中直接調用netif_rx
netif_rx★
??? if(netpoll_rx函數與把數據拿走)
??????? return
??? __skb_queue_tail(把所有收到的數據保存起來)
??? napi_schedule
??????? __napi_schedule
??????????? __raise_softirq_irqoff(NET_RX_SOFTIRQ);

在net_dev_init函數中初始化了軟中斷：
open_softirq(NET_TX_SOFTIRQ, net_tx_action, NULL);
open_softirq(NET_RX_SOFTIRQ, net_rx_action, NULL);
所以NET_RX_SOFTIRQ中斷的處理函數是net_rx_action，NET_TX_SOFTIRQ中斷的處理函數是net_tx_action。需要讓上層接收數據時，只要觸發相應的軟中斷，如__raise_softirq_irqoff(NET_RX_SOFTIRQ)。內核就會在適當時機執行do_softirq來處理pending的軟中斷。


net_rx_action★
??? n->poll = process_backlog
??????? netif_receive_skb
??????????? pt_prev->func = ip_rcv★(在這里完成了交接)
??? __raise_softirq_irqoff(NET_RX_SOFTIRQ)


發送幀
dev_queue_xmit★
??? rcu_read_lock_bh
??? if(設備有發送隊列)
??????? q->enqueue(將數據追加到發送隊列，軟中斷處理函數net_tx_action會執行真正的發送工作)
??? else
??????? dev_hard_start_xmit
??????????? dev->hard_start_xmit = el_start_xmit★
??????????????? 調用outw匯編指令發送數據，夠底層了
??? rcu_read_unlock_bh


net_tx_action★
??? __kfree_skb(釋放已發送的，此時中斷由dev_kfree_skb_irq函數發起)
??? qdisc_run
??????? __qdisc_run
qdisc_restart
??????????????? dev_hard_start_xmit★
??????????? netif_schedule
??? netif_schedule


netif_schedule★
??? __netif_schedule
??????? raise_softirq_irqoff(NET_TX_SOFTIRQ)