We are using part of a data set that looks at arsenic concentrations in Bangladesh.
VIDEO
MAKE SURE TO TAKE THE QUIZ!
Homework
Due Next Class (Monday February 26, 2024).
Take the data from class today and plot depth on the y-axis and arsenic on the x-axis. I would use ax.scatter() and I would also use the fig,ax=plt.subplots() notation. Because when you plot something versus depth you always want depth in reverse order or inverted so the 0 is at the top and the maximum is at the bottom. Use ax. and some google searching to find out how to do it. But if you stop and think about setting ranges you can change your start and stop intervals. Then the next step is to put the x-axis ticks on top and to put the label position on top. then you can set the x limits (xlim) to get the plot so it starts at 0. Also xlim could go from high to low. If something takes a list you add a [] inside the (). luckily to make this plot there is more than one way to alter all the parts. When searching for answers I would google something like “matplotlib switch axes direction”. I usually add pandas or matplotlib to the search and it should show you options.
3. Now the second graph is fancier. Take the first graph and color the points as shown below. Do this by calling scatter three times but only selecting the data you want to plot each time you call scatter. Think about the boolean calls we did in class. These are the arsenic concentrations I am coloring. Get the first graph nice and this second graph becomes easier. Do not copy my colors. These are good colors for representing the points. Choose three colors and three symbols of your own and see how they work. As a hint I would first do the less than 10 and then greater than 50. Then you can do the 10-50 which needs an “and” of some kind.
Today we are going to look further into arrays. The link (pdf=TwoDArrays-2024).
Video
Homework
Due Wednesday February 21, 2024:
Your are only allowed to add two lines of code to answer this question. Take your code from today where you read in Brian.csv and plotted the array. You can only add two more lines of code to your work from today to make the plot. Using two lines of code and your slicing nomenclature add an exclamation point. Remember in the file I had the background as 0’s and the writing as 1’s. You need to at least make a third color!. Then print it in a color of your choice using a colormap. My answer is below. Both answers are perfectly correct. I just wanted to add more color this year! To make the lines crisp I turned off interpolation. Here is the documentation. But in imshow you need to add the keyword argument interpolation=’none’ like we did in class.
2. For part 2. Make a new csv file in excel and make it so you can print your name in more than one color. So read in your file and make the plot. By plot it I mean show your name like I show my name! As a hint. I made my csv file in excel and I made the rows and columns smaller or changed the zoom so I could see the pattern I was entering. When saving a csv in excel using the ms-dos csv option seems to work the best. When making the csv file name the csv file with your name.
3. For the third part. Make an excel file with one column for x-values and three columns for y-values. Label each column. I would do ten or so rows. So approximately ten values for the x-values. For the y-columns make each distinct. You can choose your own function or pattern (straight, curved, etc) for the y-values. But make sure each line is unique. Now read in the file and plot the lines in excel and label the axes. So you need to make an excel file and use it to plot three distinct lines.
For handing in make sure to hand in your notebook, your csv file for you name, and the excel file for your lines.
We have done some line fitting and will do more (pdf=more-line-fit-2024). Half the life of a scientist is taking data, plotting it, and looking for a correlation. Plus I want to make sure you can do this in your sleep. What we are adding now is we will start to read data from a file so we can handle much larger datasets. Today we are going to read in an excel file into Pandas which is the python library for line fitting and fit a line. We are going to look at GDP versus life expectancy. A common dataset in sustainable development. This is to the website where I got the the data but I also put the excel file on GitHub.
1. You need to find some data from the web or a paper. It could be from a paper you have read. It could be from another class. Find data with at least 10 data points. I would do somewhere from 10-50 points. Do NOT use data with dates. Dates are hard. If you use time for an x-axis it should be hours or something similar but not months/days/years with slashes. We will learn how to do dates later in the semester. All other data should work.
2. Put the data in an excel file. Add your name to the name of the excel file. This way I will be able to differentiate and read all the files.
3. Read in the excel file and fit a best fit line. Remember to add a figure caption and all other parts of a figure. Add the text box showing m,b,r,p values to the graph. In the figure caption say if the result is significant.
4. Make a new sheet in your excel file that is a copy of your data. If you data had a significant correlation (p<0.05) change the data in the second sheet to make it not significant (p>0.05). Then read in the second sheet but remember you need the sheet_name keyword. Show the new correlation. If your data had no significant correlation (p>0.05) change your data to make it significant. Show this new changed plot with a best fit line, figure caption etc. I want you to manually alter your data in a second sheet and use the new data. Add the text box showing m,b,r,p values to the graph. In the figure caption say if the result is significant.
5. Make a third sheet in your excel file. Change your data again but this time take your significant correlation and multiple the y-values by -1. Now save that sheet as a csv file with your name in the title again so I can read it in when grading. Read it in and show the best fit line, figure caption etc. Add the text box showing m,b,r,p values to the graph. In the figure caption say if the result is significant. Now answer this question. Hoes does the r-value, p-value, and r-squared value compare between when you the original data and the data when you multiplied by -1. Be specific.
6. Summary
one figure of best fit line with your data
one figure of best fit line where you altered your data to make either significant or non-significant
one figure of data altered by multiplying by -1 read in from a csv file.
7. For handing in. Hand in your notebook, excel file, and csv file. The excel file should have three sheets in it.
DON’T MISS THIS NEXT PART!
8. This part will also be graded for the final 10%. As part of the class you have to comment on three of the lectures by using the comments section of the blog over the course of the semester. You do not have to comment now. You will comment as the semester proceeds. Comments can be wide ranging but need to be designed to improve the teaching or the homework given how you learn. Would you change the notebook? Would you change the homework. Is there a hint I should have given. How can you help me improve your learning and future students learning. Previous comments have already helped me to streamline the packets and homeworks. But now to make it fun you have to write an ipython script that randomly chooses your three classes. You need to randomly choose 2 classes between classes 2 and 16 and 1 class between classes 17 and 23. There are at least two ways to do this; you could use numpy.random.randint or you could use random.randint. my answer using numpy looks like
Brian will comment on classes [15 12]
Brian will comment on classes [18]
my answer using randint looks like
Brian will comment on class 15
Brian will comment on class 3
Brian will comment on class 21
Remember since you are using a random number generator everyone’s answers should come out different! If you get two of the same number re-run it to get 3 unique numbers! Plus every time you rerun the cell your numbers will change!
We are going to explore more how to do some math in iPython and how to make some plots. Today you are going to write your own program to get the best fit line (pdf=FitALine). The equations are from Harris-‘Quantitative Chemical Analysis’ (Harris-fit-a-line) (pdf=Harris-fit-a-line). I am going to give you the equation and then using numpy you will make it work. Once it is working you will then go back and get your your KNYC and KLGA data and determine the correlation. Do not look at the answers until you are done ( Fit a line answer. Don’t look until done! pdf=Fit a line answers) today. Talk to your neighbors about turning equations into code.
VIDEO
Make sure to take your quiz!
Homework 6
Homework 6 Due Wednesday February 14, 2024
Repeated from last class.
Don’t Forget you are supposed to be working on SoloLearn. Go back and make sure you started it correctly!
THIS IS THE REGULAR HOMEWORK FOR MONDAY!
You need to go and create a dataset and then fit a line to it.
You should present a hypothesis, get the data, determine the fit.
Your data and hypothesis can be anything and does not need to be “real”. you can be creative. You could interview friends to get data points. You can search online. You can make it as fun as you want. You can make something up! But you need to state a hypothesis. Have fun with it.
You need at least 5 data points that you generate from your ideas and hypotheses. Make it look good! This is not about the data. This is about the idea of presenting a hypothesis and fitting a line! make it fun! Be goofy!
Once you have the data make the graph look professional. Then you have to write a figure caption and summary sentences about your results in markdown. This should all be in one ipython notebook with nice Markdown.
For this I mean add a markdown cell below each graph explaining the result in a figure caption.
The results should be presented from both your method for fitting a line and a python method for fitting a line. This is redundant but make sure to do it! Fit the same data twice to make sure you get the same result. You should make sure that your line fitting algorithm and the python algorithm give the same answer.
For your method I mean where you did the math on your own.
For the python method I mean linregress.
The results need to be presented in the figure as a title or a text box. I like the text box!
We are going to try a new website for learning Python. It is Sololearn and here is the link for the Python Course. https://www.sololearn.com/Play/Python/ . You will need to make a login for it.
We will start by doing through Basic Concepts and Control Structures.
You can go through and just answer the questions or also try things out. It is up to you.
I am just trying to get you extra Python Practice.
This is Due on Monday October 10, 2022 (Courseworks had it correct).
I can not track a class. The picture shows what mine looked like when done.
Make sure the picture shows your login name and your name on your computer to prove it is yours.
On the assignment in courseworks you will upload the screenshot.
Regular Homework
Go back and find when you plotted the parabola and sin x. These are really nice graphs that you made.
Use input and if statements to ask the user if they would like to plot sin x or to plot the parabola
If they choose sin x use an input statement to determine the number of pi’s they want to plot and plot the appropriate graph. This way you can show many pi values.
If they choose a parabola have them enter the a,b,c of the parabola.
For example if I choose sin of x I could then choose 32.1 pi’s to plot.
For example if I choose parabola I could then choose a=1, b=-5, c=0.02
You should have all your code in one ipython box so I can just run that one box, do my entries, and then see the plots. if-statements grading rubric
Remember to use input()
To get an integer or float (remember to use them correctly, You need to choose when you think an integer or float is more appropriate) use input like
a=int(input(‘please enter a number as an integer’))
myfloat=float(input(‘please enter a number as a float’))
Install like above. But to run I open terminal. You can open terminal by typing “terminal” in the search icon. Then at the prompt type “jupyter notebook”
On a PC
I run anaconda on my PC. It has a quirk. Install it. But then my launcher always crashes. To work around.
Instead of opening installer open the anaconda command prompt. If you can’t open this, go to the file browser and open a new “Anaconda Prompt” window.
once open type “cd /” to get to root directory. This gives you access to all your directories on your hard drive. I Can’t run from here.
type “dir” to get your list of directories.
change to the directory you want to use using cd and you are ready.
now type “jupyter notebook”.
You should have notebook now open in your browser
ON THE MILBANK 222 MACS
due to technical issues we can’t permanently install anaconda so we have to install it every class. the installer is on the root level of the hard drive.
Google Anaconda Python
Click on the MAC symbol and download 3.6
Click on the Installer and then go through installation.
You should now have Anaconda with Python version 3.9 installed.
this is the ipython notebook (pdf=time-series-first-analysis_v2) to download and complete during the first class. I will bring printouts to class. It is on github and in a pdf.
Read the text and copy the examples. In the blank cells try to figure out the code!
If you need to download the notebook right click on raw and say save as. Then you have the notebook. If you get a .txt file ending change it it .ipynb
Today’s goal is to slowly work through the notebook. Take your time. I gave us plenty of time.
Take a screen shot of your certificate and then hand it in on courseworks under assignments.
The second course is the 20 hour coding course with id WPCSGD https://studio.code.org/join/WPCSGD You do not have to do the unplugged sections. This is due 2/5/24. You CANNOT join both classes at once.
Joining these classes is quirky. Use the links above. You can also first sign in to code.org then go to https://studio.code.org/join then type in the two codes above. Check with me and I will make sure I can see your progress.
Use 2 nested for loops. This is a for loop inside a for loop. Count up in the outer for loop from 0 to 9 and then at every step count back down to zero. Remember that in np.arange(start,stop,skip). You give it a start,stop, and skip. We usually put numbers in. But you don’t have to add numbers you could put a parameter in.
Go back and get the monthly precipitation from central park that you used in class. Remember that we used this data when we were learning about lists and learning how to make plots (here is the link list today and plot it). Use a for loop and sum up the monthly data. Then call the function sum on your data. It will be np.sum(precip_list). How do they compare?
Answer Question 2.
For loop sum= 45.13
sum function sum= 45.13
QUESTION 3.
Create two lists. One with the day of the week and one with the number of classes you have on that day. Use a for loop to go through your lists and printout how many classes you have each day. Then write your total number of classes.
On Sunday I have 0 Classes
On Monday I have 2 Classes
On Tuesday I have 0 Classes
On Wednesday I have 2 Classes
On Thursday I have 0 Classes
On Friday I have 0 Classes
On Saturday I have 0 Classes
In total I have 4 Classes
QUESTION 4.
Using np.arange or np.linspace and a for loop to recreate what is below. Remember that np.sin takes radians and not degrees. You can convert yourself or use the function np.deg2rad(). Also, I wrote out degrees and did not use the symbol because latex does not show up in print statements but it does on graphs. Don’t forget what you learned about format statements a few classes ago to get all the decimals correct.
The sine of 0 degrees is 0.00
The sine of 10 degrees is 0.17
The sine of 20 degrees is 0.34
The sine of 30 degrees is 0.50
The sine of 40 degrees is 0.64
The sine of 50 degrees is 0.77
The sine of 60 degrees is 0.87
The sine of 70 degrees is 0.94
The sine of 80 degrees is 0.98
The sine of 90 degrees is 1.00
The sine of 100 degrees is 0.98
The sine of 110 degrees is 0.94
The sine of 120 degrees is 0.87
The sine of 130 degrees is 0.77
The sine of 140 degrees is 0.64
The sine of 150 degrees is 0.50
The sine of 160 degrees is 0.34
The sine of 170 degrees is 0.17
The sine of 180 degrees is 0.00
Final 10%
Fun with two for loops, line breaks, new lines, and np.arange. Can you make the pattern below? Remember you can do math in np.arange.
Homework 3 Due Next Class (Wednesday September 21, 2022)
Code.org intro course through stage 15 to stay on pace
Write 50-100 letters of nonsense in a string. But in that nonsense have a secret code that spells your name with every x letter (e.g. every 5th letter).
print the whole string
use slicing to print your name
use slicing to print your name and then make it upper case
Repeat the same thing with a list but only do your first name. This is different as you are using a list. #2 above uses a string and this uses a list. It is a subtle difference but make sure to do it correct for each.
Write the list.
Use slicing to write your name
Write your name in reverse using slicing (for this to easily work the last letter of the list needs to be the last letter of your name)
reverse the list
write your name backwards using slicing on the reversed list.
with the list reversed write your name forward with slicing.
Final 10% today. Practice some more plotting and make the graph at the bottom of the page. You should be able to put together your line plotting and cosine plotting to make it. The graph below shows cosine. I am okay with an ugly B. You will need to create two and only two numpy arrays to make the B if you set all the different coordinates correctly. But there are many ways to make a B. But as a hint think about how you can define points in the B.
%matplotlib inline
import numpy as np
import matplotlib.pyplot as plt
The matplotlib inline tells the ipython notebook to show the graphs. The import import matplotlib.pyplot as plt tells python to import the plotting package so we can use the plotting functions and make great plots.
Make an x,y plot using ax.plot.
Using the links in today’s notebook for ideas.
choose your favorite shape
choose your favorite color
choose your favorite linestyle.
choose something else to change (e.g. linewidth…)
here is my example-
What are the axes missing?
Add axis labels and a title?
Make sure to use Markdown to put your name and section at the beginning of the HW file and a description of the assignment
Use Markdown to add a figure caption. This one can be brief but get in the habit.
This is my example but it is not perfect!!!!! something similar without the missing parts will get you a 50.
Final 10%: To get the final 10% of the HW credit can you make a second plot with a slope of 4 and an intercept of 1? Remember all the parts of the first graph. Also, remember copy and paste are your friends…… Also remember what slope and intercept are!
Reminder. If you want to download it from github right click on raw and choose save link as. If you get a .txt file ending change it it .ipynb and you can now open it in your jupyter notebook.
Remember that you need to take the quiz before class on courseworks! It is labeled quiz 1 and called
Quiz 1: Python Video on Directories and Printing
Notebook 1 Due beginning of Next class. (Monday January 22, 2024)
Code.org intro course through stage 5 to stay on track.
Look at the directory structure of your computer and make a place to work from when using Python (not graded, for your own organization)
Use one print statement to make the Barnard B out of b’s with no spaces in the code but you can use backslashes in the print statements. My example is below but you can make yours look nicer! Your answer does not have to look exactly like mine. But you need to use one print statement and that print statement can not have any spaces.
Use print statements and .format to answer the math questions below exactly the same way with the exact formatting. You need to set a and b and use them in your format statement. Your print statement cannot have numbers. Repeat-There are no numbers in your print/format statement. You set the a and b before the print statement.
Use Markdown in your homework to add your name and other critical information. the homework should look nice with a nice Markdown header describing the homework. This is critical. You must add your name, date, section, and assignment in markdown.
Upload your homework to courseworks. You upload the notebook. The file you upload should end in .ipynb
Watch Next Video and Take Quiz
Barnard B (Really looks more like an 8)
Your answer should be similar to but hopefully nicer than
BBBBB
B B
B B
BBBBB
B B
B B
BBBBB
Use the print function and then \n or \t or something similar to move the cursor. there should be NO SPACES in your print function!
Set
a=9.000
set b=4.000
Write the following sentence exactly. using the .format nomenclature and no numbers only variables. Use the a and b that you set in the format and print function. Use the different operators with your variables. Look at today’s handout for examples with the print and format functions. The * should be in with the formatting calls.
When 9 is divided by 4.00 the answer is *2* with a remainder of 1.0
Then to check you did it correctly. I will open your notebook. I will change your a=9 to a=11 and it should give me this sentence
“When 11 is divided by 4.00 the answer is *2* with a remainder of 3.0”
During grading I also changed a to 15 and you should get a *3*
