Python Data Science Machine Learning Lesson 4: Data Distribution Big Data

It is easier to work with small amounts of data just to understand the different concepts.

In the real world, the data sets are much bigger, since it is difficult to gather real world data, at least at the initial stages of a project.

To counter this you can generate your own data sets, in Python you can easily create your own custom data sets.

How to obtain Big Data Sets?
In order to create big data sets, we can use the Python module NumPy, which comes with a number of methods to create random data sets, of any required size.

You will have to first import numpy as under
import numpy as np
Example: Create an array containing 500 random floats between 0 and 10

now the code will be.

code

import numpy as np
x = np.random.uniform(0.0, 10.0, 500)
print(x)

the output will be

[3.31812811e+00 1.21400025e+00 1.69355085e+00 4.11614822e+00

7.36936107e-01 1.19461272e+00 4.33769571e+00 2.28522702e+00

6.26611376e+00 8.31628410e+00 4.58795362e+00 2.58560953e+00

1.44809870e+00 8.70778332e+00 1.44137436e+00 2.12419504e+00

4.46774089e+00 8.16828712e+00 3.10335164e+00 6.48104272e+00

4.47485096e+00 5.30570168e-02 6.08459923e+00 8.34122436e+00

3.27789960e+00 4.80682386e+00 7.16938232e+00 5.16431876e+00

5.60459868e+00 7.03277721e+00 2.18849835e+00 7.43054448e+00

4.10333902e+00 4.61508189e+00 1.00713560e+00 1.80441112e+00

8.95171821e+00 8.19368420e+00 1.15865484e+00 6.69515548e+00

2.91236630e+00 7.78837023e+00 5.62735997e+00 8.76746923e+00

1.52596544e+00 2.24746803e+00 7.31458379e+00 6.12165829e+00

8.83771475e+00 2.13636880e+00 7.26014631e+00 8.31344944e+00

4.03299569e+00 7.00630403e+00 7.73137514e+00 9.37487278e+00

2.40173379e+00 5.00110006e-01 8.90519053e+00 6.40863946e+00

3.92095393e+00 9.21475118e+00 2.33250572e+00 9.34688189e+00

3.78628859e+00 6.55423515e+00 2.10703951e+00 4.16847105e+00

8.66984037e+00 6.86691540e-01 8.12603265e+00 5.90970764e+00

6.23667333e+00 8.84855475e+00 2.57495632e+00 5.66385612e+00

3.73220273e+00 1.00459733e+00 5.50034399e+00 8.68784085e+00

9.43593956e+00 5.86209861e+00 3.80893349e+00 7.26965415e+00

6.96298749e+00 3.88994439e+00 1.88585930e-01 9.50300617e-01

1.65020101e+00 7.07410558e-01 7.78054900e+00 8.94859321e+00

2.36076038e+00 4.23354956e-01 7.97620476e+00 3.62515597e+00

1.43883052e+00 2.21699598e+00 6.38091035e-01 9.98748031e+00

6.41866330e+00 1.66936598e+00 6.95427057e+00 2.11322131e+00

6.34938785e+00 4.08296871e+00 5.33233275e+00 6.40814670e+00

7.59265239e+00 5.19287999e+00 4.18033911e-01 6.03151191e+00

5.56678816e+00 3.43427360e+00 6.85682000e+00 3.26764928e+00

1.45769461e+00 1.41262438e+00 9.72326241e+00 8.51880337e+00

5.17715180e-01 5.63817083e+00 9.06258973e+00 3.24324543e+00

9.52777715e+00 1.06387080e+00 3.72686170e-01 7.90139258e+00

4.07712903e+00 2.19230957e+00 1.82465152e+00 7.49756471e+00

4.48067225e+00 8.23942527e+00 8.40612321e+00 1.28252231e+00

8.41694384e+00 8.03822250e+00 9.59414945e+00 9.53671645e+00

9.94911062e+00 8.22026959e+00 4.17659283e+00 7.71536048e+00

6.16330081e+00 8.14310385e+00 8.79082653e+00 3.19668269e+00

5.92901170e+00 1.37294953e+00 3.28686809e+00 6.83450382e+00

2.36274345e+00 8.58221911e+00 1.40212829e+00 4.90884731e+00

5.76024151e+00 2.28371694e+00 2.94140614e+00 8.34651731e+00

4.52430666e+00 6.52067256e+00 2.61033935e+00 5.25046116e+00

4.95953177e+00 6.58757623e+00 4.44446802e+00 5.48949303e+00

1.83645315e+00 4.05693428e+00 2.81652107e+00 5.46508630e+00

5.25500697e+00 7.55545048e+00 9.60979485e+00 8.19844096e+00

1.84999973e+00 8.54489458e+00 6.73697938e+00 5.36148745e+00

6.47657792e+00 1.23404307e+00 8.81137593e+00 6.87653774e+00

3.04483957e+00 6.44610669e-01 6.29961299e+00 7.46869578e+00

3.47945122e-01 5.53282801e+00 4.14890743e+00 9.95399120e-01

5.99453741e+00 1.98554138e+00 7.51045986e-01 3.91332723e+00

7.21789427e+00 4.92671122e+00 2.38874210e+00 3.20062449e+00

3.72287730e+00 3.32223088e+00 5.88301732e+00 1.30930579e+00

2.99279022e+00 9.63343356e+00 5.42299774e+00 8.65953718e+00

9.28881375e+00 6.49372885e+00 4.10853651e+00 6.84329218e+00

7.13407019e+00 5.57031261e+00 8.99888103e+00 2.78930394e+00

4.32453337e+00 9.48403458e+00 4.55931541e+00 6.16007602e+00

4.58611959e+00 3.00635098e+00 5.93437186e+00 7.15878368e+00

5.44809267e+00 7.09570707e+00 8.40794929e+00 2.64235499e-01

2.10416855e+00 4.64004423e-01 8.60449941e+00 6.40331615e+00

7.02474599e+00 9.55044327e+00 3.52447024e+00 4.10480307e+00

1.24017143e-01 7.85274149e-01 6.75610279e+00 8.43731562e+00

9.87700710e+00 3.88722972e+00 1.58736944e+00 4.54224601e+00

7.40004921e+00 7.78085084e+00 5.51172204e+00 2.32744723e+00

2.95751859e-01 4.47508265e+00 3.76354374e-01 4.27140018e+00

1.37358682e+00 7.95961545e+00 2.52801059e+00 5.91755387e+00

1.62530608e+00 5.73247001e+00 6.82250391e+00 7.86031542e+00

1.08877534e+00 3.53049755e+00 8.48636657e+00 6.34582267e+00

6.85240586e+00 7.65224748e+00 4.86266244e+00 2.07245703e+00

2.77252568e-01 8.17527185e+00 6.38693386e+00 3.62933479e+00

1.57018419e+00 4.02163198e+00 3.41414154e-01 8.08477040e+00

7.31220338e+00 5.68432525e+00 7.84787596e+00 2.67638456e+00

7.34130963e-01 8.43010468e+00 3.18343212e+00 1.68019042e+00

9.91373182e-01 1.80829771e+00 3.75082444e+00 4.27163376e+00

4.84442981e-01 8.65497824e+00 1.69535564e+00 5.72704997e+00

3.33993320e+00 5.01603703e+00 3.80715807e+00 6.54843523e+00

4.80933633e+00 1.03087476e+00 1.22005432e+00 2.20462534e+00

4.53088249e+00 5.75075226e+00 7.93842905e+00 1.02523667e+00

5.00212853e+00 7.18655054e+00 3.29977012e+00 3.80147140e+00

9.79496665e+00 1.08024146e+00 5.44889544e+00 3.92630320e+00

2.13673358e+00 6.14245786e+00 6.53660909e+00 5.40998482e+00

5.19307866e+00 2.43019970e+00 5.38388153e+00 3.34806165e+00

2.26450384e-01 2.07124537e+00 7.72116527e+00 2.48079967e-01

9.02566641e+00 7.69495497e-01 5.83194696e+00 4.74730947e+00

8.39678280e+00 7.42553282e+00 4.14871272e+00 3.38211162e+00

7.39127335e+00 4.30422280e+00 1.83417794e+00 5.25797075e+00

1.24838683e+00 6.99924937e+00 2.09730877e+00 2.38792529e-01

1.22677425e+00 9.89820799e+00 1.87257487e+00 1.10988486e+00

7.51149212e+00 6.68064578e+00 5.42640051e+00 5.07128369e+00

5.07517737e+00 2.22106272e+00 3.66349931e+00 2.68757973e+00

1.84179400e+00 9.87209258e+00 2.44273057e+00 6.81989370e-01

8.66498689e+00 9.48269522e+00 7.23425623e+00 5.09401584e+00

5.75130687e+00 5.85069418e+00 5.26578082e+00 6.63654808e-01

4.97551545e+00 8.58509544e+00 4.53653093e+00 3.85130644e+00

5.26463899e+00 5.46220889e-01 2.65917757e+00 6.24838539e+00

3.71282757e-01 4.84608571e+00 1.28004162e+00 9.52529986e+00

5.56334900e+00 2.55413945e-01 5.37338201e+00 1.14747289e+00

6.33231759e+00 2.89813639e+00 6.77199384e+00 5.47981120e+00

7.68220222e+00 7.86200522e+00 9.89083616e+00 9.32717965e+00

3.61642949e+00 5.89359031e-01 2.32599190e+00 2.81929045e+00

1.17661795e+00 8.79741882e+00 2.95192198e+00 7.39414030e+00

7.94566655e+00 2.84301237e-01 8.87416252e+00 8.14588172e+00

6.33443279e+00 8.42986638e+00 4.21222217e+00 5.87175195e-01

9.89065093e+00 6.04816688e-01 2.25328412e+00 2.36990192e+00

1.51043475e+00 7.75892439e+00 8.87204369e+00 9.69088959e+00

4.08696888e+00 3.27731251e+00 4.52714615e+00 3.44667460e+00

6.44192875e+00 5.32657602e+00 4.29079324e+00 2.86369343e+00

5.35380626e+00 1.77508311e+00 2.48297553e+00 6.80304922e-01

2.42009563e+00 8.53716607e+00 5.95162326e+00 1.80876924e+00

8.39651288e+00 9.91842098e+00 3.57339096e+00 9.37686463e-01

2.17126990e+00 3.74983779e-01 6.38808114e+00 7.28642404e+00

9.53839442e+00 4.97208584e+00 8.00495342e-01 7.69878586e-01

9.81871925e+00 2.92892843e+00 5.68948781e+00 1.51924831e+00

9.69820713e+00 3.41199288e-03 9.28412289e+00 3.76975985e+00

2.54827050e+00 6.23483523e+00 4.42231261e+00 6.42837563e+00

7.01467844e+00 7.07880362e+00 7.85269866e+00 6.82275862e+00

9.37017445e+00 1.18265169e+00 7.02062771e+00 5.47430060e+00

1.57323443e+00 9.85654563e+00 6.61615524e+00 9.48457834e+00

4.87696420e+00 5.75554079e+00 7.78358351e-01 6.68179727e+00

3.35253333e+00 6.00829612e+00 3.80663633e+00 5.37089603e+00

3.67757084e+00 7.12522619e+00 9.73403051e+00 3.09869785e+00

6.64025722e+00 7.89573755e+00 5.20276317e+00 6.68289949e+00

3.78730696e+00 5.33898392e+00 3.76841072e+00 8.78438051e+00

6.83767345e+00 2.03454442e+00 7.21780132e+00 4.90741963e+00

8.61462843e+00 6.38842471e+00 1.25600582e+00 4.91399371e+00

4.66070120e+00 3.89531625e+00 6.95961949e+00 8.41696019e+00

4.87729997e+00 3.86703320e+00 7.40245406e+00 9.60711664e+00]

Important note :The values inside the array changes since all the values in the array are created randomly between 0 to 10 of size 500

Note: if you have python installed on your pc you can install numpy as under.
Open Command Prompt from the start menu.
Inside the command prompt, type
pip install numpy
press enter
This command will install numpy on your computer after which you can run on python.

How to plot a histogram of your data sets in Python?
To plot a histogram with the same data you will have to write the code as follows.

import numpy as np
import matplotlib.pyplot as plt
the code will now be as under.

import numpy as np
import matplotlib.pyplot as plt
x = np.random.uniform(0.0, 10.0, 500)
plt.hist(x, 10)
plt.show()

the output will be

Note: if you have python installed on your pc you can install matplotlib as under.
Open Command Prompt from the start menu.
Inside the command prompt, type
pip install matplotlib
press enter
This command will install matplotlib on your computer after which you can run on python.