Glad you enjoyed the article!

Orthogonal is just another word for perpendicular. Orthogonality is not really connected to correlation. Here we pick the n'th Principal Component such that it is orthogonal to all the (n-1) principal components. This is so that we can take advantage of certain nice properties that orthognal matrices have. An example of this is: the inverse of orthogonal matrices is equal to its transpose. This helps us in actually deriving the PCA algo as shown in the proof.