Network Security and Cryptography in Computer Networks: How Ciphers, Key Exchange, and TLS Actually Work

Cipher internals for computer networks: ECB vs CBC block modes, HMAC construction, square-and-multiply RSA, a Diffie-Hellman man-in-the-middle, and TLS.

KnowledgeGate Team

Exam prep & CS education

Updated 2 Sep 202611 min read

Most network security notes tell you which mechanism serves which goal and stop there. Here the mechanisms come apart: what CBC mode actually does to your blocks, how HMAC is constructed, how a man-in-the-middle captures an unauthenticated Diffie-Hellman exchange, and what each step of a TLS handshake is for. Trace these internals once on paper and every exam variant becomes a computation you can run, not a fact you memorised.

Where cryptography sits in network security

Network security pursues confidentiality, integrity, authentication, non-repudiation, and availability, against passive attackers who read traffic and active attackers who modify, replay, or impersonate. The mapping from goal to mechanism is short: ciphers give confidentiality, hashes and MACs give integrity, and signatures with certificates give authentication and non-repudiation.

That survey view is deliberately compressed here because it already has a full treatment. The goal-by-goal survey, the firewall rule table, and the classic small-prime exam drills live in Cyber and Info Security in Computer Networks: Complete Guide with Worked RSA and Diffie-Hellman Examples. This post is the mechanics half of that pair: it goes inside each mechanism.

Symmetric-key cryptography: from Caesar to AES

Symmetric cryptography uses one shared key. In a Caesar cipher with shift 3, GATE becomes JDWH: G to J, A to D, T to W, and E to H. Substitution replaces symbols and falls to frequency analysis; transposition reorders them. Stream ciphers process a stream, while block ciphers process fixed-size blocks.

Cipher

Block size

Key size

Rounds and structure

DES

64 bits

56 effective bits

16-round Feistel network

3DES

64 bits

Two or three DES keys

Applies DES three times

AES

128 bits

128, 192, or 256 bits

10, 12, or 14 rounds; not Feistel

A block cipher also needs a mode of operation. Electronic Code Book encrypts every block independently, so identical plaintext blocks give identical ciphertext blocks and the outline of a structured image survives encryption. Cipher Block Chaining XORs each block with the previous ciphertext block, starting from a random initialisation vector, so the repeats disappear. When a question offers ECB, it is almost always the wrong answer.

Two consequences follow from the chaining. CBC encryption is sequential, and a reused initialisation vector leaks whether two messages begin the same way, which is why the vector must be fresh and random every time. Error behaviour differs too: a corrupted ciphertext block in CBC garbles that block completely but only flips the matching bits in the next one, because decryption XORs against the received ciphertext, not a computed value.

identical plaintext blocks producing identical ECB ciphertext blocks vs chained CBC output

Symmetric systems share one problem the cipher itself cannot solve: every pair of users needs its own secret key, and that pairwise growth in key count is what motivates public-key cryptography.

RSA by hand: square-and-multiply on any exam modulus

RSA follows a fixed recipe:

  1. Choose primes p and q.

  2. Calculate n = p x q and phi(n) = (p-1)(q-1).

  3. Choose e coprime to phi(n).

  4. Find d such that e x d is congruent to 1 modulo phi(n).

  5. Publish (e, n) and keep (d, n) private. Encrypt with C = M^e mod n; decrypt with M = C^d mod n.

The skill worth building is the arithmetic that makes the last step cheap. Two tools cover every exam modulus: square-and-multiply, which assembles a large exponent from repeated squarings, and the negative-representative trick, which swaps a residue r for r - n when the negative number is smaller to square. Watch both run once end to end.

Take p = 5 and q = 11. Then n = 55 and phi(n) = 4 x 10 = 40. Choose e = 3, which is coprime to 40. The private exponent is d = 27, because 3 x 27 = 81 = 2 x 40 + 1, so 3 x 27 is congruent to 1 modulo 40.

Encrypting M = 7 is direct: C = 7^3 mod 55 = 343 mod 55, and 343 = 6 x 55 + 13, so C = 13.

Decryption asks for 13^27 mod 55. Write 27 = 16 + 8 + 2 + 1, so 13^27 = 13^16 x 13^8 x 13^2 x 13. Build the powers by squaring, reducing after every step:

  • 13^2 = 169 = 3 x 55 + 4, so 13^2 is 4 mod 55.

  • 13^4 = 4^2 = 16.

  • 13^8 = 16^2 = 256 = 4 x 55 + 36, so 13^8 is 36 mod 55.

  • 13^16: here the negative representative pays off. 36 is -19 mod 55, and 19^2 = 361 is far easier than 36^2 = 1296. Since 361 = 6 x 55 + 31, 13^16 is 31 mod 55.

Multiply the powers you need: 31 x 36 = 1116 = 20 x 55 + 16, then 16 x 4 = 64, which is 9 mod 55, then 9 x 13 = 117 = 2 x 55 + 7. So 13^27 mod 55 = 7 and the message is recovered.

Nothing here depended on 55. Given any n, e, and d, the same moves apply: decompose the exponent into powers of two, square with a reduction at every step, switch to a negative representative whenever it shrinks the square, and reduce to the range 0 to n-1 at the very end. That final reduction is the step people skip; -28 is not an answer to a mod 55 question, 27 is.

RSA square-and-multiply flow for p=5, q=11: key generation gives n=55, e=3, d=27; encrypting M=7 yields C=13; the squaring chain 13^2, 13^4, 13^8, 13^16 with reductions recombines as 13^27 mod 55 = 7

Diffie-Hellman under a man-in-the-middle: the full walkthrough

Diffie-Hellman lets two parties agree a symmetric key over a public channel: with a public prime p and generator g, Alice sends g^a mod p, Bob sends g^b mod p, and each raises the value received to their own private exponent. The discrete logarithm keeps the private exponents safe, so the mathematics is sound. The protocol's weakness is different: neither message says who it came from.

Run the attack with p = 17 and g = 3. Alice picks a = 9. Since 3^2 = 9, 3^4 = 81 is 13 mod 17, and 3^8 = 13^2 = 169 is 16 mod 17, she sends A = 3^9 = 16 x 3 = 48, which is 14 mod 17. Bob picks b = 7 and sends B = 3^7 = 13 x 9 x 3: here 13 x 9 = 117 is 15 mod 17, and 15 x 3 = 45 is 11 mod 17, so B = 11.

Mallory sits on the wire with her own exponent m = 5 and computes 3^5 = 13 x 3 = 39, which is 5 mod 17. She intercepts A = 14 before it reaches Bob and sends him 5 instead, and intercepts B = 11 before it reaches Alice and sends her 5 as well. Neither side can tell, because a Diffie-Hellman public value is just a number.

Alice now computes what she believes is the shared secret: 5^9 mod 17. Squaring: 5^2 = 25 is 8, then 5^4 = 8^2 = 64 is 13, which is -4 mod 17, so 5^8 = (-4)^2 = 16, which is -1, and 5^9 = -1 x 5 = -5, which is 12 mod 17. Mallory computes the same key from her side as A^m = 14^5. Since 14 is -3 mod 17, (-3)^5 = -243, and 243 = 14 x 17 + 5, so 14^5 is -5, which is 12. Mallory and Alice now share K1 = 12.

Bob computes 5^7 mod 17 = 5^4 x 5^2 x 5 = 13 x 8 x 5. First 13 x 8 = 104 = 6 x 17 + 2, then 2 x 5 = 10. Mallory matches it with B^m = 11^5. Since 11 is -6 mod 17, (-6)^2 = 36 is 2 mod 17, so (-6)^4 = 4 and (-6)^5 = 4 x (-6) = -24, which is 10 mod 17. Mallory and Bob share K2 = 10.

Alice encrypts with 12, Mallory decrypts with 12, reads or rewrites the message, re-encrypts with 10, and Bob decrypts with 10. Both endpoints see a clean working channel; there is one secret per leg and Mallory holds both. No key size fixes this, because the mathematics ran correctly on every leg. What was missing is authentication of the exchange itself, and that is exactly why TLS signs its key exchange with a certificate before any session key is trusted.

Hashes, MACs, signatures, and certificates

A cryptographic hash maps any input to a fixed-size digest. MD5 gives 128 bits and is broken; SHA-1 gives 160 bits and is deprecated; SHA-256 gives 256 bits. Secure hashes need preimage, second-preimage, and collision resistance. An attacker who alters a message can also replace an unprotected hash.

Mechanism

Integrity

Authentication

Non-repudiation

Hash alone

Limited, unless the reference digest is protected

No

No

MAC with a shared key

Yes

Yes, between key holders

No

Digital signature

Yes

Yes

Yes

A MAC combines the message with a shared secret. HMAC is the standard construction, hashing twice over keyed variants of the message so that the naive key-plus-message form cannot be extended. Either key holder could have produced the tag, so a MAC settles integrity and authentication between them and gives no non-repudiation against a third party. A digital signature hashes the message and signs the digest with the sender's private key, so only the sender could have produced it. A certificate authority signs the binding between an identity and a public key, and verification follows a chain to a root CA trusted by the browser or operating system. A worked RSA signature, computed and verified with small numbers, is in Cyber and Info Security in Computer Networks.

The TLS handshake step by step

TLS sits between an application protocol and its transport service; HTTPS is HTTP over TLS, normally on port 443 rather than port 80. Everything in this post meets in the handshake: an authenticated key exchange defeats the man-in-the-middle above, and the session it establishes hands the data to a symmetric cipher in a safe mode. Underneath it all runs TCP, whose role is covered in TCP vs UDP: Transport Layer Explained. The handshake in order:

  1. Client hello and parameter negotiation. The client offers its TLS versions and cipher suites plus a fresh random value, and the server picks the strongest set both sides support. Skip this and the peers cannot even agree which cipher and hash the rest of the handshake uses; negotiate it badly and a downgrade attack pushes both sides onto broken algorithms.

  2. Server certificate presented. The server sends a certificate binding its domain name to its public key. Skip this and the client is running the key exchange with whoever answered, which is precisely Mallory's position in the walkthrough above.

  3. Client verifies the CA chain. The client checks the certificate's signature chain up to a root certificate authority it already trusts, and checks that the domain matches. Skip this and a certificate is just a self-declared name anyone can mint; verification is what makes the previous step mean something.

  4. Authenticated Diffie-Hellman establishes session keys. Both sides run an ephemeral Diffie-Hellman exchange, and the server signs its key-exchange values with the certified key. Skip the signature and you are back to unauthenticated Diffie-Hellman with K1 and K2; skip ephemerality and a server key stolen later exposes every recorded session.

  5. Symmetric record protection carries the data. The agreed secret is expanded into symmetric session keys, and every record is encrypted and integrity-protected with them. Public-key operations are far too slow for bulk traffic, which is why the handshake only ever bootstraps a symmetric session.

IPsec: transport mode, tunnel mode, AH and ESP

IPsec works at the network layer. Transport mode protects the payload between end hosts; tunnel mode wraps the entire original IP packet, which is what site-to-site VPNs use. It carries two protocols: the Authentication Header gives integrity and origin authentication but no confidentiality, while Encapsulating Security Payload also encrypts, which is why deployments normally choose ESP.

The scope contrast is worth one clean sentence: TLS protects one application connection, IPsec protects a network path between two endpoints, and a firewall enforces boundary policy without encrypting anything. Firewall types and rule ordering are covered in the survey guide, Cyber and Info Security in Computer Networks.

Cryptography traps that cost marks

  • Wrong totient: for n = 55, phi(n) = (5-1)(11-1) = 40, never 55 - 1 = 54. Subtracting one from n only works when n itself is prime.

  • ECB mode: encrypting each block on its own leaves identical plaintext blocks identical in the ciphertext. Chaining modes with an initialisation vector do not.

  • The naive MAC: hash(key + message) fails because standard hash constructions allow length extension, so an attacker can append data and extend the tag without knowing the key. HMAC hashes twice over keyed variants of the message exactly to block this.

  • Sign slips in the negative-representative method: a dropped minus sign quietly corrupts the whole chain. The check is mechanical: track the sign at every squaring and multiplication, and reduce the final answer to the range 0 to n-1 before writing it down.

  • AH versus ESP: the Authentication Header authenticates and nothing more. If the question mentions confidentiality, the answer is ESP, because AH never encrypts.

  • Overtrusting HTTPS: the certificate certifies that the key belongs to the domain, not that the organisation behind the domain is honest. A phishing site with a valid certificate for its own domain still shows the padlock.

How GATE and interviews test the mechanics

The mechanics here map to four recurring question shapes: mode-of-operation MCQs where ECB is the tempting wrong answer, true-or-false rows lifted straight off the hash, MAC, and signature matrix above, handshake-ordering questions asking which TLS step comes before which, and one-liners separating AH from ESP. Interviews ask the same material openly: walk through a TLS handshake in order, or explain why unauthenticated Diffie-Hellman fails. For the scope-and-strategy discussion of which papers ask what and how much time the topic deserves, use the survey guide, Cyber and Info Security in Computer Networks.

After re-running the RSA and Diffie-Hellman arithmetic above with numbers of your own, test the transport layer these protocols ride on with Computer Networks TCP and UDP transport layer MCQs.

The short version

  • A block cipher is only as good as its mode: ECB leaks plaintext structure, CBC chains it away with an initialisation vector.

  • HMAC exists because naive keyed hashing extends: hash(key + message) falls to length extension, so HMAC hashes twice.

  • Square-and-multiply with negative representatives makes any exam modulus tractable on paper; always reduce to 0 to n-1 at the end.

  • Unauthenticated Diffie-Hellman falls to a man in the middle holding one key per leg; certificates in the TLS handshake are the fix.

  • AH authenticates; ESP authenticates and also encrypts. IPsec protects a path, TLS protects a connection.

For complete Computer Networks coverage, continue with Zero to Hero. Then use the GATE Test Series to practise the same decisions under timed conditions.