Author Topic: Learning to Code  (Read 3544 times)

0 Members and 1 Guest are viewing this topic.

Offline werecow

  • Cryptobovinologist
  • Frequent Poster
  • ******
  • Posts: 3014
  • mooh
Re: Learning to Code
« Reply #105 on: March 01, 2017, 06:11:59 AM »
Yep, exactly.

EDIT: This video may clarify things a bit:

« Last Edit: March 01, 2017, 09:06:01 AM by werecow »
Mooohn!

Offline The Latinist

  • Cyber Greasemonkey
  • Technical Administrator
  • Reef Tank Owner
  • *****
  • Posts: 9103
Re: Learning to Code
« Reply #106 on: March 01, 2017, 11:31:28 AM »
So thinking about it a little more, I think I can explain the difference in time between the two ways of  looping. Tell me if this makes sense or if I'm way off. The important thing to remember is that in Python "for" is actually always "foreach"; that is, "for" always and only iterates over collections.  When you're iterating with a "for i in range" statement, you're actually creating a list of integers, then iterating over that list.  Prior to 3.0, you were creating the whole list first, then iterating (unless you used xrange); since 3.0, you're generating the list as a stream and iterating as it's generated. There is not a difference in complexity between "for i in range(len(listname)" and "for name in listname" because in both cases you're doing exactly the same thing. Well, except for the len(listname) part, which is of O(1) complexity and done just once.  And if you want to take every third, you're doing the same thing with "for index, element in enumerate(my_list[::3])" that you would be doing with "for i in range(0, len(my_list), 3)": you're iterating over the entire list, taking every third element.  The difference between the two ways of looping is this: when you use "for i in range()" you have to do a separate list.get to get the actual value of my_list, whereas you've already got it right there in a tuple when you use enumerate.  So the upshot is this:

def i_loop():
    for i in range(0, len(my_list), 3):
        temp_variable = (my_list)

Four steps, all complexity O(1), steps 2-4 all repeated for each in loop:

1. Find the length of my_list.
2. Generate a stream of the integers and iterates over it, taking only every third element.
3. For each i, gets the list element my_list
4. Assigns result of my_list to temp_variable

def e_loop():
    for index, element in enumerate(my_list[::3]):
        temp_variable = element

Two steps, both O(1) complexity repeated for each in loop:

1. Generates a stream of tuples and iterates over it, taking only every third element.
2. Assigns element to temp_variable using tuple assignment.
I would like to propose...that...it is undesirable to believe in a proposition when there is no ground whatever for supposing it true. — Bertrand Russell

Offline werecow

  • Cryptobovinologist
  • Frequent Poster
  • ******
  • Posts: 3014
  • mooh
Re: Learning to Code
« Reply #107 on: March 01, 2017, 01:15:22 PM »
Well, like I noted before, I don't notice a lot of difference between "for i in range(0, len(x), 3)" and "for i,j in enumerate(x[::3])". The difference between my measure() and measure3() above was tiny and was probably just noise. The initial difference (in the version without "x[::3]") can easily be explained by the fact that enumerate went over the entire list, doing a "if i % 3 == 0" check.

Also, for some reason, every time I read the word measure now I hear it in the way a pirate might say "treasure"...  Arrrgh, may-sure! I may have been watching too much Pirates of the Carribean and Black Sails lately.
« Last Edit: March 01, 2017, 01:20:32 PM by werecow »
Mooohn!

Offline Johnny Slick

  • "Goddammit, Slick."
  • Poster of Extraordinary Magnitude
  • **********
  • Posts: 11887
  • Fake Ass Skeptic
Re: Learning to Code
« Reply #108 on: March 01, 2017, 01:51:18 PM »
THIS IS THE MOST OFFENSIV THING IVE EVER READ ON THESE FORA MEASURE AND TREASURE ALREADY RHYME THE END
Speak what you think now in hard words, and to-morrow speak what to-morrow thinks in hard words again, though it contradict every thing you said to-day.

- Ralph Waldo Emerson

Offline werecow

  • Cryptobovinologist
  • Frequent Poster
  • ******
  • Posts: 3014
  • mooh
Re: Learning to Code
« Reply #109 on: March 01, 2017, 01:55:04 PM »
Of course they do. May-sure / Tray-sure. Arrrr.
Mooohn!

Offline The Latinist

  • Cyber Greasemonkey
  • Technical Administrator
  • Reef Tank Owner
  • *****
  • Posts: 9103
Re: Learning to Code
« Reply #110 on: March 01, 2017, 02:57:36 PM »
werecow: I've timed the two types of loop repeatedly, and for i, j in enumerate() is consistently almost twice as fast as for i in range(). I'm not sure why you're seeing similar numbers.

That said, for most purposes it won't make much difference. My only point remains that manipulating indices directly in for loops is not pythonic, and is typical in code written by people who come from other languages.
« Last Edit: March 01, 2017, 03:00:37 PM by The Latinist »
I would like to propose...that...it is undesirable to believe in a proposition when there is no ground whatever for supposing it true. — Bertrand Russell

Offline werecow

  • Cryptobovinologist
  • Frequent Poster
  • ******
  • Posts: 3014
  • mooh
Re: Learning to Code
« Reply #111 on: March 01, 2017, 04:16:39 PM »
werecow: I've timed the two types of loop repeatedly, and for i, j in enumerate() is consistently almost twice as fast as for i in range(). I'm not sure why you're seeing similar numbers.

There must be some difference between our implementations (or our interpreters) that we're missing, because I'm still getting only a minor difference at best (and if anything, the enumerate version is slightly slower). Are you on python 3? I'm on 2.7.13.

EDIT:
Yep, I think it's in the interpreter:

python 2.7:
Code: [Select]
('range:', (0.04399991035461426, [(0, 0), (3, 3), (6, 6), (9, 9), (12, 12), (15, 15), (18, 18), (21, 21), (24, 24), (27, 27), (30, 30), (33, 33), (36, 36), (39, 39), (42, 42), (45, 45), (48, 48), (51, 51), (54, 54), (57, 57), (60, 60), (63, 63), (66, 66), (69, 69), (72, 72), (75, 75), (78, 78), (81, 81), (84, 84), (87, 87), (90, 90), (93, 93), (96, 96), (99, 99)]))
('xrange:', (0.04200005531311035, [(0, 0), (3, 3), (6, 6), (9, 9), (12, 12), (15, 15), (18, 18), (21, 21), (24, 24), (27, 27), (30, 30), (33, 33), (36, 36), (39, 39), (42, 42), (45, 45), (48, 48), (51, 51), (54, 54), (57, 57), (60, 60), (63, 63), (66, 66), (69, 69), (72, 72), (75, 75), (78, 78), (81, 81), (84, 84), (87, 87), (90, 90), (93, 93), (96, 96), (99, 99)]))
('enumerate:', (0.05900001525878906, [(0, 0), (3, 3), (6, 6), (9, 9), (12, 12), (15, 15), (18, 18), (21, 21), (24, 24), (27, 27), (30, 30), (33, 33), (36, 36), (39, 39), (42, 42), (45, 45), (48, 48), (51, 51), (54, 54), (57, 57), (60, 60), (63, 63), (66, 66), (69, 69), (72, 72), (75, 75), (78, 78), (81, 81), (84, 84), (87, 87), (90, 90), (93, 93), (96, 96), (99, 99)]))
('enumerate, no correction:', (0.046000003814697266, [(0, 0), (1, 3), (2, 6), (3, 9), (4, 12), (5, 15), (6, 18), (7, 21), (8, 24), (9, 27), (10, 30), (11, 33), (12, 36), (13, 39), (14, 42), (15, 45), (16, 48), (17, 51), (18, 54), (19, 57), (20, 60), (21, 63), (22, 66), (23, 69), (24, 72), (25, 75), (26, 78), (27, 81), (28, 84), (29, 87), (30, 90), (31, 93), (32, 96), (33, 99)]))

Python 3.5:
Code: [Select]
range: (0.07490825653076172, [(0, 0), (3, 3), (6, 6), (9, 9), (12, 12), (15, 15), (18, 18), (21, 21), (24, 24), (27, 27), (30, 30), (33, 33), (36, 36), (39, 39), (42, 42), (45, 45), (48, 48), (51, 51), (54, 54), (57, 57), (60, 60), (63, 63), (66, 66), (69, 69), (72, 72), (75, 75), (78, 78), (81, 81), (84, 84), (87, 87), (90, 90), (93, 93), (96, 96), (99, 99)])
enumerate: (0.056334495544433594, [(0, 0), (3, 3), (6, 6), (9, 9), (12, 12), (15, 15), (18, 18), (21, 21), (24, 24), (27, 27), (30, 30), (33, 33), (36, 36), (39, 39), (42, 42), (45, 45), (48, 48), (51, 51), (54, 54), (57, 57), (60, 60), (63, 63), (66, 66), (69, 69), (72, 72), (75, 75), (78, 78), (81, 81), (84, 84), (87, 87), (90, 90), (93, 93), (96, 96), (99, 99)])
enumerate, no correction: (0.05000185966491699, [(0, 0), (1, 3), (2, 6), (3, 9), (4, 12), (5, 15), (6, 18), (7, 21), (8, 24), (9, 27), (10, 30), (11, 33), (12, 36), (13, 39), (14, 42), (15, 45), (16, 48), (17, 51), (18, 54), (19, 57), (20, 60), (21, 63), (22, 66), (23, 69), (24, 72), (25, 75), (26, 78), (27, 81), (28, 84), (29, 87), (30, 90), (31, 93), (32, 96), (33, 99)])

(First number is the time in seconds. Enumerate, no correction does not correct the indices by multiplying by 3.)

(click to show/hide)
« Last Edit: March 01, 2017, 04:52:54 PM by werecow »
Mooohn!

Offline The Latinist

  • Cyber Greasemonkey
  • Technical Administrator
  • Reef Tank Owner
  • *****
  • Posts: 9103
Re: Learning to Code
« Reply #112 on: March 01, 2017, 09:08:34 PM »
Yes, I'm using Python 3.6. Let's try something in both:

Code: [Select]
import random, string
from timeit import timeit

my_list = []

def create_list(length):
    for i in xrange(length):
        my_list.append(''.join(random.choice(string.lowercase) for j in range(10)))


def i_loop():
    for i in xrange(0, len(my_list), 3):
        temp = (my_list[i])


def e_loop():
    for index, item in enumerate(my_list[::3]):
        temp = (item)

create_list(10000)

time_i = timeit(i_loop, number=10000)

time_e = timeit(e_loop, number=10000)

print time_i

print time_e

Python 3.6.0:

i_loop:  6.555159816998639
e_loop:  4.144582626002375

Python 2.7.10:

('i_loop: ', 2.1695828437805176)
('e_loop: ', 3.0389139652252197)

I didn't realize there was such a difference between 2.7 and 3.6 in speed.

(Also clearly forgot to change my print functions to statements.)

ETA: Had a hunch that it was default long integers that was making the difference, so I changed the list to 10,000 random 10-character strings.  It brought the two closer, thought 2.7 is still faster:

3.6:

i-loop: 2.968450517997553
e-loop: 3.916395651001949

2.7:

i-loop: 2.27516889572
e-loop: 2.95699310303

Using xrange instead of range speeds things up slightly in 2.7:

i-loop: 2.11893796921
e-loop: 2.86101222038

Interesting that the i loop is now faster even in 3.6.  Will have to play with this some more.

ETA2:

(click to show/hide)

i-loop: 1.35195112228
e-loop: 1.37893414497

(click to show/hide)

i-loop: 2.4153426230041077
e-loop: 1.9637810919957701

Aside: I'm very much not trying to prove any kind of point, here.  I'm just trying to learn as much as I can (and this exercise has already taught me a lot about what kinds of operations are expensive in Python 3.6 vs 2.7, etc.
« Last Edit: March 01, 2017, 10:08:38 PM by The Latinist »
I would like to propose...that...it is undesirable to believe in a proposition when there is no ground whatever for supposing it true. — Bertrand Russell

Offline The Latinist

  • Cyber Greasemonkey
  • Technical Administrator
  • Reef Tank Owner
  • *****
  • Posts: 9103
Re: Learning to Code
« Reply #113 on: March 06, 2017, 01:24:55 AM »
Playing around with pexpect (a pure Python module that implements most of the functions of expect) to automate some maintenance tasks.  I periodically have to enable error-disabled ports in our school's media center.  It's not a huge task, but I always have to look up which port is associated with which interface and login to three different switches to accomplish the task. It's a bit annoying.  Or it was: I now have a script that prompts for ports to enable and ssh password and then runs the whole process from there.  It's kind of awesome.

Next up is a script to enable and disable the students' WiFi with a single command (currently the only practical way to accomplish it is through a web interface).
« Last Edit: March 06, 2017, 01:27:30 AM by The Latinist »
I would like to propose...that...it is undesirable to believe in a proposition when there is no ground whatever for supposing it true. — Bertrand Russell

Offline The Latinist

  • Cyber Greasemonkey
  • Technical Administrator
  • Reef Tank Owner
  • *****
  • Posts: 9103
Re: Learning to Code
« Reply #114 on: March 06, 2017, 08:56:10 PM »
Script written to automate the process of disabling and enabling student WiFi.  For this one I decided I'd like command line arguments, so I played with argparse.  Very handy.  Now if I run student_wifi.py -e/--enable it enables WiFi, and if I run student_wifi.py -d/--disable it disables it (prompting for the SSH password using getpass.getpass in each case).  I also set up some graceful error handling in case it couldn't open the SSH connection (likely due to the fact that I've forgotten to connect to the VPN).

This is a lot of fun!
I would like to propose...that...it is undesirable to believe in a proposition when there is no ground whatever for supposing it true. — Bertrand Russell

Offline Pusher Robot

  • Frequent Poster
  • ******
  • Posts: 2203
  • Do you have stairs in your house?
Re: Learning to Code
« Reply #115 on: March 06, 2017, 11:32:20 PM »
Once you start, it's hard to stop.  But, if anyone questions whether it's worth it, just consult The Chart: https://xkcd.com/1205/
A novice was trying to fix a broken Lisp machine by turning the power off and on.
Knight, seeing what the student was doing, spoke sternly: “You cannot fix a machine by just power-cycling it with no understanding of what is going wrong.”
Knight turned the machine off and on.
The machine worked.

Offline ScepticalBadger

  • Off to a Start
  • *
  • Posts: 12
Re: Learning to Code
« Reply #116 on: April 15, 2017, 06:19:23 PM »
I would say Python is a good choice.  I have been doing a fair bit of python lately.  It is good enough for the likes of Dropbox, Twitter and facebook,  so you know it scales well.  It is designed to be friendly and easy to learn.  It has Many modules for different functions.  Popular amongst data scientists. 
I have dabbled with many different languages from Pascal, Ada, C C++ C# Java, Pearl.  Python is the one I stuck with the longest.

If you are going to go Python,  learn Python 3, not 2.7.  Although people claim more modules for 2.7, it is no longer in active development and the plan is to depricate it in 2020.  3 has many fixes and improvements over 2.7.

Online Andrew Clunn

  • Poster of Extraordinary Magnitude
  • **********
  • Posts: 15035
  • Aspiring Super Villain
Re: Learning to Code
« Reply #117 on: April 16, 2017, 12:18:52 AM »
Once you start, it's hard to stop.  But, if anyone questions whether it's worth it, just consult The Chart: https://xkcd.com/1205/

By that measure, doing things to get a better night's rest is like the most productive thing ever.
Banned from both Hacker News and the xkcd forum.

 

personate-rain
personate-rain