A novel approach has been developed to detect a micropipette tip under dry objective. The tip detection algorithm consists of four steps:
1. Mask image construction: In a microscope image, there might be several moving objects other than a micropipette; There might also be intensity changes which are not really associated with any moving ob- ject but are created by illumination changes or noise. It is important to recognize the pixels of the image associated with the specified mi- cropipette. To achieve this goal, we define a mask image which is a binary image with white pixels on and around the specified micropipette and black pixels otherwise. To construct such a mask image, we use a spatio-temporal filtering technique. A micropipette is moved to within a small range of its current location in the lateral direction (orthogonal to the projection of micropipette’s central axis in the image plane) with a fixed frequency fv(Hz). Images are captured at a rate of 30Hz and a
band-pass temporal frequency filter is applied to the stream of images, pixel by pixel. The filter passes frequencies around fv(Hz) and rejects
other frequencies. The pixels that pass this filter should specify the points on and around the specified micropipette, because it is the only object vibrating at this frequency. To choose an appropriate frequency
fv, there are different considerations to be taken into account:
• Our sampling rate is 30Hz so we should have fv <15.0 to satisfy
• The micromanipulators’ maximum operation frequency is 12.57Hz
due to a 79.5msdelay for execution of each command. So we should havefv <12.57.
• The frequency should be away from the frequency ranges of any mechanical vibrations or optical noises affecting the equipment and images.
• Higher frequencies result in shorter time of detection, but affect the life of the micromanipulator.
Considering all the above-mentioned limitations, we have chosen fv =
5.0Hz and we only run it for 5 cycles which takes 1.0sec. This tempo- ral filtering results in a gray-scale image Ig
n where n (n ∈ {1,2,3,4}
in this case) is the index for the specified micropipette. This gray- scale image is then thresholded. The threshold level is adjusted to
τn=max ςn,12max(Ing)
, whereςn= ¯Ing+ 2.5σ(Ing) determines the noise
level, ¯Ing is the average intensity ofIng andσ(Ing) is the standard deviation of the pixel intensities in Ing. The result is a binary image Inb. Using ς
helps to avoid false detection when there is no significant motion. When 1
2max(I
g
n) ≤ ςn, the threshold is set to ςn which indicates that there is
no significant motion with the specified frequency which usually means that micropipette is out of the field of view, totally out of focus or there is not enough illumination. In such cases, the user is asked to check if micropipette is visible and then repeat the procedure.
The resulting binary image Inb might have some pixel noise and also the detected region may have some gaps. A morphological opening and closing is then applied to the binary image Ib
n to remove pixel noise and
fill in the gaps in the binary image [23].
Inm = (Inb ◦Mno)•Mnc (4.9) The result is a binary imageInm which is called the mask image. Mno, Mnc
and closing operations.
2. Orientation Detection: To detect the orientation of the central axis of a micropipette, we use a technique based on theHough transform [24]. In an arbitrary situation, two images are captured: In(0,−δy) which is the
image captured when the micropipette is moved−δy(µm) with respect to its original position in a direction orthogonal to its central axis parallel to focal plane and In(0, δy) which is captured when the micropipette
is moved δy(µm) in the reverse direction. After capturing these two images, we construct two new images by thresholding the differences of these images: Iny− =τ(In(0,−δy)−In(0,+δy)) Iny+ =τ(In(0,+δy)−In(0,−δy)) (4.10)
A Progressive Probabilistic Hough Transform (P P HT) [25] is then ap- plied to the resulting binary images Iny− and Iny+ to detect the orienta- tion of the micropipette. The orientation of the strongest line segment detected in each of the images is selected and the bisector of these line segments is determined as the orientation of the central axis of the mi- cropipette as shown in Figure 4.5 (c) and (d). Due to the cone-like shape of the micropipette, the two line segments detected inIny− andIny+ will not be parallel, we use this fact to find the direction that points towards the micropipette tip.
3. Tip proximity detection: To detect the proximity of the tip, we use a similar approach as in the previous step. We obtain two images In(0,0)
and In(−δx,0) representing the images captured when the micropipette
is in its original position and when it is moved back δx(µm) along its central axis, respectively. δxis selected based on the pixel size of the im- age. Then we calculate the absolute value of intensity difference between these two images and mask it byInm to make sure it is only showing pix- els in the proximity of the nth micropipette tip. Then we threshold the
result to obtain a binary imageInx:
Inx =τ(|In(0,0)−In(−δx,0)|.I
m
n ) (4.11)
Then a morphological opening and closing is applied to Inx. The result is a binary image which is white in a small proximity of the tip (as shown in Figure 4.5(d)) and black all over the image.
4. Accurate tip detection: To search for the exact location of the tip, the binary image Inx is searched along the direction of the micropipette detected in previous steps and the ultimate pixel with intensity of 1 is detected as the micropipette tip.
All of this procedure is illustrated in Figure 4.5.
Figure 4.5: Tip detection algorithm – results: (a) original image, (b) mask im- age (negated for better visibility), (c) overlappedIny− and Iny+, (d) proximity of the micropipette tip as detected in Inx, negated to increase visibility, and (e) the detected line segment along the central axis of the micropipette.