1. Firstly, I think that you need to have at least some kind of conceptual model in mind (or, better, on paper). This model should guide you in your exploratory data analysis (EDA). A presence of a dependent variable (DV) in the model means that in your machine learning (ML) phase later in the analysis you will deal with so called supervised ML, as opposed to unsupervised ML in the absence of an identified DV.

2. Secondly, EDA is a crucial part. IMHO, EDA should include multiple iterations of producing descriptive statistics and data visualization, as you refine your understanding about the data. Not only this phase will give you valuable insights about your datasets, but it will feed your next important phase - data cleaning and transformation. Just throwing your raw data into a statistical software package won't give much - for any valid statistical analysis, data should be clean, correct and consistent. This is often the most time- and effort-consuming, but absolutely necessary part. For more details on this topic, read these nice papers: http://vita.had.co.nz/papers/tidy-data.pdf (by Hadley Wickham) and http://cran.r-project.org/doc/contrib/de_Jonge+van_der_Loo-Introduction_to_data_cleaning_with_R.pdf (by Edwin de Jonge and Mark van der Loo).

3. Now, as you're hopefully done with EDA as well as data cleaning and transformation, your ready to start some more statistically-involved phases. One of such phases is exploratory factor analysis (EFA), which will allow you to extract the underlying structure of your data. For datasets with large number of variables, the positive side effect of EFA is dimensionality reduction. And, while in that sense EFA is similar to principal components analysis (PCA) and other dimensionality reduction approaches, I think that EFA is more important as it allows to refine your conceptual model of the phenomena that your data "describe", thus making sense of your datasets. Of course, in addition to EFA, you can/should perform regression analysis as well as apply machine learning techniques, based on your findings in previous phases.

Finally, a note on software tools. In my opinion, current state of statistical software packages is at such point that practically any major software packages have comparable offerings feature-wise. If you study or work in an organization that have certain policies and preferences in term of software tools, then you are constrained by them. However, if that is not the case, I would heartily recommend open source statistical software, based on your comfort with its specific programming language, learning curve and your career perspectives. My current platform of choice is R Project, which offers mature, powerful, flexible, extensive and open statistical software, along with amazing ecosystem of packages, experts and enthusiasts. Other nice choices include Python, Julia and specific open source software for processing big data, such as Hadoop, Spark, NoSQL databases, WEKA. For more examples of open source software for data mining, which include general and specific statistical and ML software, see this section of a Wikipedia page: http://en.wikipedia.org/wiki/Data_mining#Free_open-source_data_mining_software_and_applications.

UPDATE: Forgot to mention Rattle (http://rattle.togaware.com), which is also a very popular open source R-oriented GUI software for data mining.

1. Firstly, I think that you need to have at least some kind of conceptual model in mind (or, better, on paper). This model should guide you in your exploratory data analysis (EDA). A presence of a dependent variable (DV) in the model means that in your machine learning (ML) phase later in the analysis you will deal with so called supervised ML, as opposed to unsupervised ML in the absence of an identified DV.

2. Secondly, EDA is a crucial part. IMHO, EDA should include multiple iterations of producing descriptive statistics and data visualization, as you refine your understanding about the data. Not only this phase will give you valuable insights about your datasets, but it will feed your next important phase - data cleaning and transformation. Just throwing your raw data into a statistical software package won't give much - for any valid statistical analysis, data should be clean, correct and consistent. This is often the most time- and effort-consuming, but absolutely necessary part. For more details on this topic, read this nice paper (by Hadley Wickham) and this (by Edwin de Jonge and Mark van der Loo).

3. Now, as you're hopefully done with EDA as well as data cleaning and transformation, your ready to start some more statistically-involved phases. One of such phases is exploratory factor analysis (EFA), which will allow you to extract the underlying structure of your data. For datasets with large number of variables, the positive side effect of EFA is dimensionality reduction. And, while in that sense EFA is similar to principal components analysis (PCA) and other dimensionality reduction approaches, I think that EFA is more important as it allows to refine your conceptual model of the phenomena that your data "describe", thus making sense of your datasets. Of course, in addition to EFA, you can/should perform regression analysis as well as apply machine learning techniques, based on your findings in previous phases.

Finally, a note on software tools. In my opinion, current state of statistical software packages is at such point that practically any major software packages have comparable offerings feature-wise. If you study or work in an organization that have certain policies and preferences in term of software tools, then you are constrained by them. However, if that is not the case, I would heartily recommend open source statistical software, based on your comfort with its specific programming language, learning curve and your career perspectives. My current platform of choice is R Project, which offers mature, powerful, flexible, extensive and open statistical software, along with amazing ecosystem of packages, experts and enthusiasts. Other nice choices include Python, Julia and specific open source software for processing big data, such as Hadoop, Spark, NoSQL databases, WEKA. For more examples of open source software for data mining, which include general and specific statistical and ML software, see this section of a Wikipedia page.

UPDATE: Forgot to mention Rattle, which is also a very popular open source R-oriented GUI software for data mining.

UPDATE: Forgot to mention Rattle (http://rattle.togaware.com), which is also a very popular open source R-oriented GUI software for data mining.