Triple nested for loop python

I want to reduce the time of this triple anycodings_loops nested for-loop:

dtz = []

for i in range(0, len(Profile)):
    dtz.append(pd.DataFrame(columns=['Strecke', 'Ziel', 'Distanz']))

Columns = ['Strecke', 'Ziel', 'Distanz']
for i in range(0, len(Profile)):
    for col in Columns:
        dtz[i][col] = np.zeros(len(Viertel))

for i in range(0, len(Profile)):
    for j in range(0, len(Fahrstartzeit[i])):
        for k in range(0, len(Viertel)):
            if Viertel[k] >= Fahrstartzeit[i][j] and
                Viertel[k] < Fahrendezeit[i][j]:
                dtz[i]['Strecke'][k-1] = Profile[i]['Strecke'][j]
                dtz[i]['Ziel'][k-1] = Profile[i]['Ziel'][j]

The information is this. There are 4.000 anycodings_loops 'Profile'(For-loop-->i), each of this anycodings_loops 'Profile' have four(4) columns with anycodings_loops different lenght (For-loop-->j): anycodings_loops 'Fahrstartzeit', 'Fahrendezeit', 'Strecke', anycodings_loops 'Ziel'. The vector 'Viertel' anycodings_loops (For-loop-->k)has intervals of 15 minutes anycodings_loops and is this one:

Viertel = pd.date_range(start='2012-01-01', end='2012-12-31 23:45:00', freq='15min').to_pydatetime

So I want to take into account the departure anycodings_loops - 'Fahrstartzeit' and arrival time anycodings_loops 'Fahrendezeit'of each trip and write the in anycodings_loops the vector 'Viertel'. That means, if a trip anycodings_loops starts at 07:13 and ends at 8:35, the anycodings_loops information Profile[i]['Strecke'][j] in the anycodings_loops DataFrame dtz[i]['Strecke'][k-1] when the anycodings_loops condition is meet. The condition is:

if Viertel[k] >= Fahrstartzeit[i][j] and
                    Viertel[k] < Fahrendezeit[i][j]:

For example if anycodings_loops Fahrstartzeit[i][j]=02.01.2012 07:02:00 and anycodings_loops Fahrendezeit[i][j]=02.01.2012 08:43:00 it anycodings_loops will be written in anycodings_loops Profile[i]['Strecke'][j]=dtz[i]['Strecke'][k-1] anycodings_loops for all Viertel[k] that are in between those anycodings_loops times:

Viertel[k]=02.01.2012 07:15:00
Viertel[k]=02.01.2012 07:30:00
Viertel[k]=02.01.2012 07:45:00
Viertel[k]=02.01.2012 08:00:00
Viertel[k]=02.01.2012 08:15:00
Viertel[k]=02.01.2012 08:30:00

Of course this has to be done without much anycodings_loops effort, because the vectors are huge.

How can I make this simpler?

Not as elegant as it should be

Photo by Johannes Plenio on Unsplash

We all know that Python is an elegant programming language. But everything has weaknesses. Sometimes Python is not as elegant as it should be.

For example, when we need to break out of nested loops as follows:

for a in list_a:
for b in list_b:
if condition(a,b)…

This article describes how to break out of nested loops in Python.

  • How to write nested loops in Python
  • Break out of nested loops with else and continue
  • Break out of nested loops with a flag variable
  • Avoid nested loops with itertools.product()
  • Speed comparison

See the following article for the basic usage of the for loop in Python.

  • for loop in Python (with range, enumerate, zip, etc.)

How to write nested loops in Python

In Python, nested loops (multiple loops) are written as follows. Blocks are represented by indents in Python, so just add more indents.

l1 = [1, 2, 3]
l2 = [10, 20, 30]

for i in l1:
    for j in l2:
        print(i, j)
# 1 10
# 1 20
# 1 30
# 2 10
# 2 20
# 2 30
# 3 10
# 3 20
# 3 30

When break is executed in the inner loop, it only exits from the inner loop and the outer loop continues.

for i in l1:
    for j in l2:
        print(i, j)
        if i == 2 and j == 20 :
            print('BREAK')
            break
# 1 10
# 1 20
# 1 30
# 2 10
# 2 20
# BREAK
# 3 10
# 3 20
# 3 30

Break out of nested loops with else and continue

In Python's for loop, you can use else and continue in addition to break.

  • for loop in Python (with range, enumerate, zip, etc.)

You can break all loops with else and continue.

for i in l1:
    for j in l2:
        print(i, j)
        if i == 2 and j == 20:
            print('BREAK')
            break
    else:
        continue
    break
# 1 10
# 1 20
# 1 30
# 2 10
# 2 20
# BREAK

The code with explanation is as follows.

for i in l1:
    print('Start outer loop')

    for j in l2:
        print('--', i, j)
        if i == 2 and j == 20:
            print('-- BREAK inner loop')
            break
    else:
        print('-- Finish inner loop without BREAK')
        continue

    print('BREAK outer loop')
    break
# Start outer loop
# -- 1 10
# -- 1 20
# -- 1 30
# -- Finish inner loop without BREAK
# Start outer loop
# -- 2 10
# -- 2 20
# -- BREAK inner loop
# BREAK outer loop

When the inner loop ends normally without break, continue in the else clause is executed. This continue is for the outer loop, and skips break in the outer loop and continues to the next cycle.

When the inner loop ends with break, continue in the else clause is not executed. In this case, break in the outer loop is executed.

As a result, whenever the inner loop ends with break, break in the outer loop is also executed.

The idea is the same even if the number of loops increases. An example of a triple loop is as follows.

l1 = [1, 2, 3]
l2 = [10, 20, 30]
l3 = [100, 200, 300]

for i in l1:
    for j in l2:
        for k in l3:
            print(i, j, k)
            if i == 2 and j == 20 and k == 200:
                print('BREAK')
                break
        else:
            continue
        break
    else:
        continue
    break
# 1 10 100
# 1 10 200
# 1 10 300
# 1 20 100
# 1 20 200
# 1 20 300
# 1 30 100
# 1 30 200
# 1 30 300
# 2 10 100
# 2 10 200
# 2 10 300
# 2 20 100
# 2 20 200
# BREAK

Break out of nested loops with a flag variable

The above way of using else and continue may be difficult to understand for those unfamiliar with Python.

Adding a flag variable may make the code easier for many to understand.

In the condition that the inner loop ends with break, set the flag to True, and in the outer loop, set break according to the flag.

Double-loop:

l1 = [1, 2, 3]
l2 = [10, 20, 30]

flag = False
for i in l1:
    for j in l2:
        print(i, j)
        if i == 2 and j == 20:
            flag = True
            print('BREAK')
            break
    if flag:
        break
# 1 10
# 1 20
# 1 30
# 2 10
# 2 20
# BREAK

Triple-loop:

l1 = [1, 2, 3]
l2 = [10, 20, 30]
l3 = [100, 200, 300]

flag = False
for i in l1:
    for j in l2:
        for k in l3:
            print(i, j, k)
            if i == 2 and j == 20 and k == 200:
                flag = True
                print('BREAK')
                break
        if flag:
            break
    if flag:
        break
# 1 10 100
# 1 10 200
# 1 10 300
# 1 20 100
# 1 20 200
# 1 20 300
# 1 30 100
# 1 30 200
# 1 30 300
# 2 10 100
# 2 10 200
# 2 10 300
# 2 20 100
# 2 20 200
# BREAK

You can avoid nested loops with itertools.product().

  • Cartesian product of lists in Python (itertools.product)

You can use itertools.product() to get all combinations of multiple lists in one loop and get the same result as nested loops.

import itertools

l1 = [1, 2, 3]
l2 = [10, 20, 30]

for i, j in itertools.product(l1, l2):
    print(i, j)
# 1 10
# 1 20
# 1 30
# 2 10
# 2 20
# 2 30
# 3 10
# 3 20
# 3 30

Since it is a single loop, you can simply break under the desired conditions.

for i, j in itertools.product(l1, l2):
    print(i, j)
    if i == 2 and j == 20:
        print('BREAK')
        break
# 1 10
# 1 20
# 1 30
# 2 10
# 2 20
# BREAK

Adding the argument of itertools.product(), you can execute the process corresponding to more multiple loops.

l1 = [1, 2, 3]
l2 = [10, 20, 30]
l3 = [100, 200, 300]

for i, j, k in itertools.product(l1, l2, l3):
    print(i, j, k)
    if i == 2 and j == 20 and k == 200:
        print('BREAK')
        break
# 1 10 100
# 1 10 200
# 1 10 300
# 1 20 100
# 1 20 200
# 1 20 300
# 1 30 100
# 1 30 200
# 1 30 300
# 2 10 100
# 2 10 200
# 2 10 300
# 2 20 100
# 2 20 200
# BREAK

Note

In itertools.product(), the process for the element is always executed for all combinations.

In the following example, the multiplication is performed 9 times for both i and j.

for i, j in itertools.product(l1, l2):
    x = i * 2 + j * 3
    print(i, j, x)
# 1 10 32
# 1 20 62
# 1 30 92
# 2 10 34
# 2 20 64
# 2 30 94
# 3 10 36
# 3 20 66
# 3 30 96

In the case of nested loops, the process for the outer loop is executed by the number of outer elements.

In the following example, the multiplication for the variable i is only 3 times.

for i in l1:
    temp = i * 2
    for j in l2:
        x = temp + j * 3
        print(i, j, x)
# 1 10 32
# 1 20 62
# 1 30 92
# 2 10 34
# 2 20 64
# 2 30 94
# 3 10 36
# 3 20 66
# 3 30 96

Speed comparison

The result of measuring the execution time of each way with the magic command %%timeit of Jupyter Notebook is shown. Note that it cannot be measured if executed as Python code.

  • Measure execution time with timeit in Python

Please note that the results will differ depending on the number of elements and the number of for loops to be nested.

Take a triple loop with 100 elements as an example.

import itertools

n = 100
l1 = range(n)
l2 = range(n)
l3 = range(n)

x = n - 1

%%timeit
for i in l1:
    for j in l2:
        for k in l3:
            if i == x and j == x and k == x:
                break
        else:
            continue
        break
    else:
        continue
    break
# 43 ms ± 1.33 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)

%%timeit
flag = False
for i in l1:
    for j in l2:
        for k in l3:
            if i == x and j == x and k == x:
                flag = True
                break
        if flag:
            break
    if flag:
        break
# 45.2 ms ± 3.42 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)

%%timeit
for i, j, k in itertools.product(l1, l2, l3):
    if i == x and j == x and k == x:
        break
# 55.8 ms ± 458 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)

Using else, continue and adding flag variables are roughly equivalent, and itertools.product() is slow.

However, in some cases, itertools.product() is more suitable because it improves the readability of code even if it is slow. You should use it depending on the situation.

How does a 3 nested for loop work?

When a loop is nested inside another loop, the inner loop runs many times inside the outer loop. In each iteration of the outer loop, the inner loop will be re-started. The inner loop must finish all of its iterations before the outer loop can continue to its next iteration.

Why would you not want to have 3 nested for loops in a function?

It's a practice to avoid as much as possible because the number of nested loops and efficiency are directly connected. If you have 1 nested loop complexity of algorithm is on average O(n)2 and 2 nested loops complexity increase to O(n)3.

How many nested for loops is too many?

Microsoft BASIC had a nesting limit of 8. @Davislor: The error message refers to the stack size in the compiler, which is also a program, and which is recursively processing the nested-looped construct.

What are the 3 loops in Python?

The three types of loop control statements are: break statement. continue statement. pass statement.