A dimension is a space in which a given number of coordinates are required to specify the location of a point.
Eg, the transpose of a vector V = [1 2 3 4 5] specifies a point in a 5-dimensional space. It's worth remembering that vectors and dimensions DO NOT have to refer to points in space. The geometric picture is just one way of visualizing an abstract set of relationships that can be generalized to a number of different, fairly esoteric objects.
For example, the function f(x) = 3x+1 is actually a vector on which you can do transformations similar to matrix multiplication (fun fact: the function f(x) = e^x is an eigenvector under the derivative transformation!).
So in answer to your question, I don't think language is a dimension per say, but rather, a practically infinite dimensional vector space.
It's not hard to construct such a space. Let's call it L-space and an homage-a Terry Pratchett. The first thing we'd have to do is define a basis for L (for those of you who don't remember your linear algebra a basis is a unique set of vectors from which the entire space can be spanned).
Our basis (call it B) for L could be every individual word in the language, in no particular order. We could have:
B = [apple, dog, fruit, bunny, and, cucumber, lugubrious, quixotic, fluffy...]
So if you wanted to define the a sentence (S1) like "the quick brown fox jumped over the lazy dog," you would treat that as an operation on your basis vector. You'd end up with something like this:
S1 = [0_apple, 1_dog, 0_fruit, 0_bunny, 0_and ... 1_quick, 0_marzipan, 1_jumped, 0_mighty...1_brown...]
Basically you would construct S1 by putting a 0 before every word NOT in "the quick brown fox jumped over the lazy dog" and a 1 before any word that was in it. If we had a sentence (S2) like "Buffalo buffalo Buffalo buffalo buffalo buffalo Buffalo buffalo" you would start with the same basis B but after applying transformation S2 you'd end up with
S2 = [0_apple, 0_dog, 0_fruit...8_buffalo...]
You get the idea.
Now, this is an incomplete space in which to define language, after all, it doesn't include any non-word symbols (although we could simply append them to the end our our basis vector). It also doesn't encode any information about the ORDER or words. For that you would need a new space.
You could create a new vector space, where the basis vector was every possible sentence, which would allow you to define individual sentences, but you run into the same problem as before when you try and define paragraphs.
So: is language a dimension? No. Is language a vector space? Yes - one in which ideas or collections of ideas serve as points. Can we easily construct the vector space? No.
NOW
There's been some talk about binary, which I think is going a little bit off the rails. Binary is not a language, nor is it a coordinate system. Binary is just a way of representing numbers in a base-2 system. It's really no different then our base-10 system, besides the fact that it uses fewer digits.
Blessings
~ND