Lukse.lt

I was always fascinated by lightning. Long time ago, being a teenager I shoot lightning strikes many times with some degree of success. At that time I was using my dad’s old Zenit camera. While lightnings are always the same, equipment advances very rapidly. Today I will talk about spotting interesting frames from video and detecting exact moment of strike using Python.

This is relatively simple task for a script – measure difference between two frames and if it exceeds predetermined value assume this is lightning. Key idea – lightning produces lots of light and alters many pixels. So, being Python and computer vision enthusiast, I wrote quick and dirty OpenCV script to find specific frames in long video to spot lightning strikes.

It happened that script is pretty fast and processes 15minutes of raw 1080p@60 footage from GOPRO camera in 8 minutes on i7 CPU.

Besides spotting interesting pictures it saves motion values to CSV file for further analysis using other tools. Here is how motion time-chart looks like. In general this is fast motion detection script without localization feature.

Setup was out of the box – GOPRO 3 black action camera without any harness. It was just recording plain video. All processing was done on computer.

Motion chart shows 12 possible lightning strikes

Results

Animated GIF – Lightning strike to TV tower

Animated GIF – lightning in Pašilaičiai

Animated GIF – Lightning over Gariūai

Lightning strike to TV tower

Making GIF with ImageMagick

convert.exe -resize 640x360 -layers OptimizeTransparency +map -delay 15 -loop 0 gif\*.jpg gif\out.gif

Quick and dirty Python script for motion detection

import sys
import cv2
import cv2.cv as cv
import numpy as np
from PIL import Image
import progress
import time

SCALE = 0.5
NOISE_CUTOFF = 5
BLUR_SIZE = 3

start = time.time()

def count_diff(img1, img2):
    small1 = cv2.resize(img1, (0,0), fx=SCALE, fy=SCALE)
    small2 = cv2.resize(img2, (0,0), fx=SCALE, fy=SCALE)
    #cv2.imshow('frame', small2)
    #cv2.waitKey(1)
    diff = cv2.absdiff(small1, small2)
    diff = cv2.cvtColor(diff, cv2.COLOR_RGB2GRAY)
    frame_delta1 = cv2.threshold(diff, NOISE_CUTOFF, 255, 3)[1]
    frame_delta1_color = cv2.cvtColor(frame_delta1, cv2.COLOR_GRAY2RGB)
    delta_count1 = cv2.countNonZero(frame_delta1)

    return delta_count1

filename = sys.argv[1]
video = cv2.VideoCapture(filename)

nframes = (int)(video.get(cv.CV_CAP_PROP_FRAME_COUNT))
width = (int)(video.get(cv.CV_CAP_PROP_FRAME_WIDTH))
height = (int)(video.get(cv.CV_CAP_PROP_FRAME_HEIGHT))
fps= (int)(video.get(cv.CV_CAP_PROP_FPS))

frame_count = 0

print "[i] Frame size: ", width, height
print "[i] Total frames:", nframes
print "[i] Fps:", fps

fff = open(filename+".csv", 'w')

flag, frame0 = video.read()
treshold = int(sys.argv[2])
strikes = 0

p0 = progress.AnimatedProgressBar(end=nframes, width=80)
for f in xrange(nframes-1):
    p0 + 1
    p0.show_progress()

    flag, frame1 = video.read()
    diff1  = count_diff(frame0, frame1)
    name = filename+"_%06d.jpg" % f

    if diff1 > treshold:
        cv2.imwrite(name, frame1)
        strikes = strikes + 1

        #small = cv2.resize(frame1, (0,0), fx=SCALE, fy=SCALE)
        #cv2.imshow('frame', small)
        #cv2.waitKey(1)

    text = str(f)+', '+str(diff1)
    #print text
    fff.write(text  + '\n')
    fff.flush()
    frame0 = frame1

fff.close()
print
print '[i] Strikes: ', strikes
print '[i] elapsed time:', time.time() - start

Notes

GOPRO CMOS sensor has rolling shutter and has no manual exposure settings. That’s why many frames are split into dark and light areas. There a good thing to use ND filter to make exposure longer. In this way whole lightning strike will fit into frame and picture will be higher quality. Of course best option is to use global shutter camera, but it is difficult to find this kind of camera.