How can I compare the grammatical complexity between two texts using their sentences dependency length?

Question

This is a continuation to the following thread.

I have two texts, common English texts such as news articles and informative texts versus a technical textbook. I want to compare the grammatical complexity between those texts using their sentences dependency length in order to conclude whether they both have the same level of complexity or not. I was thinking about using the p-value as an evidence against the null hypothesis. Here is how the data would look like:

Text 1

ID  Dependency Length   Sentence Length
0   13                  7
1   5                   3
2   20                  8

Text 2

ID  Dependency Length   Sentence Length
0   8                   5
1   10                  7
2   14                  7

By the way, I am using python.

Answer 1

If your goal is only to determine whether the level of complexity is the same between these two sources based on these particular features, yes you can simply compare the distributions with a Student's t-test if the distributions are normal or a Wilcoxon test if they are not.

Spoiler alert: it's very likely that they are different. A statistical significance test doesn't give you much information, it's usually much more useful to try to quantify the level of complexity, but it's also much harder. Based on your previous linked question, you might be interested to read about text complexity/readability, quite a lot of research has been done on this topic (e.g. here, here, or there, among many others). There are also a few general tutorials apparently (e.g. this or that). I also know that there have been a number of readability/complexity metrics proposed in the literature, but I don't have references.

Note that looking at the features from the text is unlikely to be very useful on its own. Probably you will need either some kind of corpus annotated by complexity, or to use a metric which has been proved to correlate with sentence complexity.