Lightning strikes and Python

2015-05-10 13:58 | Categories: In English, Nuotraukos, Projektai, Time-lapse, Užrašai, Video

Gp027304.mp4_045126_I was always fascinated by lightning. Long time ago, being a teenager I shoot lightning strikes many times with some degree of success. At that time I was using my dad’s old Zenit camera. While lightnings are always the same, equipment advances very rapidly. Today I will talk about spotting interesting frames from video and detecting exact moment of strike using Python.

 

This is relatively simple task for a script – measure difference between two frames and if it exceeds predetermined value assume this is lightning. Key idea – lightning produces lots of light and alters many pixels. So, being Python and computer vision enthusiast, I wrote quick and dirty OpenCV script to find specific frames in long video to spot lightning strikes.

It happened that script is pretty fast and processes 15minutes of raw 1080p@60 footage from GOPRO camera in 8 minutes on i7 CPU.

Besides spotting interesting pictures it saves motion values to CSV file for further analysis using other tools. Here is how motion time-chart looks like. In general this is fast motion detection script without localization feature.

Setup was out of the box – GOPRO 3 black action camera without any harness. It was just recording plain video. All processing was done on computer.

Untitled

Motion chart shows 12 possible lightning strikes

Results

zaibai_i_tv_boksta

Animated GIF – Lightning strike to TV tower

out9

Animated GIF – lightning in Pašilaičiai

out6

Animated GIF – Lightning over Gariūai

Gp017304.mp4_036444_

Lightning strike to TV tower

Making GIF with ImageMagick

convert.exe -resize 640x360 -layers OptimizeTransparency +map -delay 15 -loop 0 gif\*.jpg gif\out.gif

Quick and dirty Python script for motion detection

import sys
import cv2
import cv2.cv as cv
import numpy as np
from PIL import Image
import progress
import time

SCALE = 0.5
NOISE_CUTOFF = 5
BLUR_SIZE = 3

start = time.time()

def count_diff(img1, img2):
    small1 = cv2.resize(img1, (0,0), fx=SCALE, fy=SCALE)
    small2 = cv2.resize(img2, (0,0), fx=SCALE, fy=SCALE)
    #cv2.imshow('frame', small2)
    #cv2.waitKey(1)
    diff = cv2.absdiff(small1, small2)
    diff = cv2.cvtColor(diff, cv2.COLOR_RGB2GRAY)
    frame_delta1 = cv2.threshold(diff, NOISE_CUTOFF, 255, 3)[1]
    frame_delta1_color = cv2.cvtColor(frame_delta1, cv2.COLOR_GRAY2RGB)
    delta_count1 = cv2.countNonZero(frame_delta1)

    return delta_count1

filename = sys.argv[1]
video = cv2.VideoCapture(filename)

nframes = (int)(video.get(cv.CV_CAP_PROP_FRAME_COUNT))
width = (int)(video.get(cv.CV_CAP_PROP_FRAME_WIDTH))
height = (int)(video.get(cv.CV_CAP_PROP_FRAME_HEIGHT))
fps= (int)(video.get(cv.CV_CAP_PROP_FPS))

frame_count = 0

print "[i] Frame size: ", width, height
print "[i] Total frames:", nframes
print "[i] Fps:", fps

fff = open(filename+".csv", 'w')

flag, frame0 = video.read()
treshold = int(sys.argv[2])
strikes = 0

p0 = progress.AnimatedProgressBar(end=nframes, width=80)
for f in xrange(nframes-1):
    p0 + 1
    p0.show_progress()

    flag, frame1 = video.read()
    diff1  = count_diff(frame0, frame1)
    name = filename+"_%06d.jpg" % f

    if diff1 > treshold:
        cv2.imwrite(name, frame1)
        strikes = strikes + 1

        #small = cv2.resize(frame1, (0,0), fx=SCALE, fy=SCALE)
        #cv2.imshow('frame', small)
        #cv2.waitKey(1)

    text = str(f)+', '+str(diff1)
    #print text
    fff.write(text  + '\n')
    fff.flush()
    frame0 = frame1

fff.close()
print
print '[i] Strikes: ', strikes
print '[i] elapsed time:', time.time() - start

Notes

GOPRO CMOS sensor has rolling shutter and has no manual exposure settings. That’s why many frames are split into dark and light areas. There a good thing to use ND filter to make exposure longer. In this way whole lightning strike will fit into frame and picture will be higher quality. Of course best option is to use global shutter camera, but it is difficult to find this kind of camera.

  1. 2015-05-10 21:12

    It looks fantastic!!! :)

  2. Regiscruzbr
    2015-05-12 02:14

    Great project!!!

    Are you thinking about keep publishing the shots here on your blog?

    I’m also fascinated by lightning, I can stay hours and hours looking at it. We have pretty nice ones here in Brazil.

    Thanks for share.

    • Saulius Lukšė
      2015-05-12 06:25

      Hi Regiscruzbr, thank you for your support! Of course I will publish new shots on my blog. Waiting for new lightning storm. Feel free to share your pictures, Brazil is an amazing place and you should have impressive pictures too.

  3. Ivan
    2015-05-12 06:46

    Great! Here some suggestions: you can also try to capture only frames that are lighter then previous, also OpenCV stores pixels in BGR format (it affects how grayscale image gets computed), blurring images that needs to be compared can also reduce a lot of noise on your graphs.

    small1 = blur(small1,5)
    small2 = blur(small2,5)
    gray1 = cvtColor(small1, cv2.COLOR_BGR2GRAY)
    gray2 = cvtColor(small2, cv2.COLOR_BGR2GRAY)
    mask = compare(gray1, gray2, CMP_GT)
    diff = absdiff(gray1,gray2)
    diff = min(diff,mask)
    frame_delta = cv2.threshold(diff, NOISE_CUTOFF, 255, 3)[1]
    delta_count = cv2.countNonZero(frame_delta)

    • Saulius Lukšė
      2015-05-12 06:47

      Hi Ivan, thank you for suggestion. Will try later today!

  4. Julius
    2015-05-14 16:19

    Awesome project! I have a lot of various videos with lightning strikes from GoPro and other cameras that I would love to use your script on. I have been trying to get the script to run, I have added OpenCV,numphy and Pillow of the modules to Python 2.7 on Windows but I am missing the import progress module could you point me in the right direction where to find it?
    Thanks

  5. stein
    2016-06-10 14:07

    How do I import my video’s to the script.
    (Sorry, I’m a still a noob with programming)

    • Saulius Lukšė
      2016-06-10 14:50

      python script takes two arguments first is video file name, second is threshold. For example [main.py video.avi 10000] threshold has to be chosen by guess depending on results.

  6. Nathan
    2016-06-25 19:33

    Can’t get it to work. Error that pops up:

    p0 = progress.AnimatedProgressBar(end=nframes, width=80)
    AttributeError: ‘module’ object has no attribute ‘AnimatedProgressBar’

    • Saulius Lukšė
      2016-06-25 22:04

      Hi Nathan,

      I used old animated progress bar library for more visual indication. Feel free to remove it. Lines:

      import progress

      p0 = progress.AnimatedProgressBar(end=nframes, width=80)

      p0 + 1
      p0.show_progress()

  7. Daniel
    2016-07-08 23:28

    Hey, this is great! I would love to be able to do this as well, but I have very basic coding experience. I have Python and all the modules loaded, but I get the following message when I try to execute the script:

    File “C:/Users/DDavidson/Desktop/Test.py”, line 28, in
    filename = sys.argv[1]

    IndexError: list index out of range

    I have the script and my video both on the desktop. Script is named Test.py and Video is named Test.mp4

    Any ideas?

    • Saulius Lukšė
      2016-07-09 07:28

      Hi Daniel, I think you have not provided threshold value. Your command line should look like “Test.py Test.mp4 10000”.

      • Daniel
        2016-07-13 19:00

        Okay, I got that to run, but it is not really processing the mp4 file.
        [i] Frame size: 0 0
        [i] Total frames: 0
        [i] Fps: 0

        [i] Strikes: 0
        [i] elapsed time 0.00999999046326

        and the csv file it generated is blank. Should the video file be avi instead or should that even matter?

        • Saulius Lukšė
          2016-07-13 19:19

          Hi Daniel, yes I guess should be converted or search for instructions on Python forums how to make PIL read MP4 files.

          • Travis
            2017-04-25 05:27

            For someone new to python and trying to understand the script with no comments, how would I do that…raw read from the h.264 file would be great to have.

          • Saulius Lukšė
            2017-04-25 06:24

            h.264 in some cases might not work straight of the box in opencv. Might need recompiling from sources. I suggest using this snippet to convert video file to separate frames: ffmpeg -i video.avi image%06d.jpg

  8. Hert
    2017-08-12 21:02

    This would be awesome if you actually provided some instructions. I am a photographer, not a programmer and I have no bloody clue what to make of this script.

    • Saulius Lukšė
      2017-08-14 14:03

      I guess I should prepare standalone scripts for easier use. What OS do you use?

      • Daniel
        2017-08-27 05:20

        I would love to have a standalone script for this! I am using Windows 10 Pro x64

  9. Craig Manktelow
    2018-06-29 20:42

    Firstly, thank you so much, I know it’s been some time since you did this, so I adapted my version for cv3, however this is producing amazing results. One thing I did find, which I kind of expected, is I have a video where I moved the camera half way through recording and it flagged every frame. I’m sure there’s some way around this but for me it’s brilliant, so thank you again!!!! so much

You must be logged in to post a comment.
TOP