SEP Focus improvements
ClosedPublic

Authored by murveit on Apr 12 2020, 6:54 AM.

Details

Summary

This PR should improve the HFR calculations with SEP for the focus module and for displaying HFR for those that want to track it in subs.
It is not the end-all, but should be an improvement. There is still work to do.

  • Updated and added to tests for the fitssepdetector.cpp
  • Removed saturated stars from the HFR calculation, if possible.
  • Sped up the SEP star detector 2X when used with full exposures.
  • Improved SEP star detector quality, especially for focus/hfr, by (1) changing the deblend constant, and (2) removing the largest stars
  • Fixed two bugs in the SEP star detector (1) major issue, ushorts being treated as shorts, and (2) a memory leak for (rare) float images
  • Added logging for HFR when calculated
  • Fixed bug so that Mark Stars now works
  • Improved drawing of circles for Mark Stars, now uses star radius, instead of flakey star width.
  • Updated the fitsviewer status to include the HFR when calculated
  • Increased a threshold in SEP extract.c which previously would fail on images, e.g. like M42, where there was a lot of area over background.

Not done, but related:

  • Should improve the MarkStars UI so that HFR could be calculated without star annotation on display, and could turn on on first image.
  • API to star detection methods should be generalized (should wait on rlancaste's investigations of SEP best practices)
  • Some display of HFR trends.
Test Plan

Basic test added to testfitsdata for SEP, run that.
Load a .fits file into fitsviewer. Select view->MarkStars, for a full-exposure sub, you should see ~100 stars circled with an HFR displayed in the status bar.
Load another image and the HFR should continue to be displayed.
Similarly, one could make a capture sequence job with, say 20 5s simulator exposures. After the first exposure completes and the fitsviewer displays
the image, go to the view menu and turn on Mark Stars, then the rest of the images should display with stars circled and HFR displayed in the status bar.
This way of turning on the star/HFR detection should be improved in a future PR.

Diff Detail

Repository
R321 KStars
Branch
sep-focus-improvements (branched from master)
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 25297
Build 25315: arc lint + arc unit
murveit created this revision.Apr 12 2020, 6:54 AM
Restricted Application added a project: KDE Edu. · View Herald TranscriptApr 12 2020, 6:54 AM
Restricted Application added a subscriber: kde-edu. · View Herald Transcript
murveit requested review of this revision.Apr 12 2020, 6:54 AM

That's excellent! I have several points in the comments. I would also like you to add more FITS fixtures captured with the CCD Simulator or real pictures, with various FWHM values (use the focuser simulator) to verify your improvements.

kstars/fitsviewer/fitssepdetector.cpp
135

What if 0.8 of the source set means 0 sources? Is the remaining algorithm safe? Shouldn't we discard sources based on their distance in sigma units instead? (that value may not be available, sure)

186

Yes! Good spot!

kstars/fitsviewer/fitsview.cpp
789

I disagree : it was like this before, but small stars ellipses become offset because of rounding issues on the pixel center.

Also would it be possible to report on the performance improvement with numbers? I'm curious!

Thanks. I'll discuss your other comments in a bit, but first here is detail on the speedup. The basic idea is that previously the code was calling sep_flux_radius, an expensive call, on all detections unnecessarily. Instead, I first sort of the detected stars' oval size, since we are only keeping the largest stars, and only call sep_flux_radius on the stars we intend to keep. I checked and sorting by HFR after calling sep_flux_radius on all stars vs first sorting by oval radii and only calling sep_flux_radius on those give almost the same set of stars. 

The extent of the speedup is directly related to the number of detections from sep_extract, and how much larger that is than the desired number of stars (I kept that at 100). So, on the "double stars" image, with 11K detections, the speedup is an order of magnitude. On the cropped focus image, with only 3 stars detected, there is no speedup. In general, on full-frame images, we detect more than 100 stars, even with short exposures, so there is a speedup. For my full-frame images it tends to be about a factor of 2.

Here are the images I've tested with:
fullframe: a full-frame 4656x3520 gocus image, green 5s
cropframe: 192x192 focus image, green 5s
doublestars: a fullframe sub from an RGB camera, 300s exposure
sunflower: a fullframe 4656x3520 sub from a monochrome camera, 240s Red filter
They can be found in https://drive.google.com/drive/folders/1jlwxCh7J27_jmQZugD8RgB1ExY276XPP?usp=sharing

Mac SPEEDUP DISABLED

QBenchmark on fixture: 252ms
doublestars. SEP: 9.470s (11208 detected 100 final)
sunflower    SEP: 0.864s (680 detected 100 final)
fullframe:   SEP: 0.646s (427 detected 100 final)
crop frame   SEP: 0.006s (3 detected 3 final)

Mac WITH SPEEDUP

QBenchmark on fixture: 119ms (273 detected 100 final)
doublestars. SEP: 0.939s (11208 detected 100 final)  
sunflower    SEP: 0.473s (680 detected 100 final)
fullframe:   SEP: 0.410s (427 detected 100 final)
crop frame   SEP: 0.005s (3 detected 3 final)

RPi4 4Gb Raspbian SPEEDUP DISABLED

QBenchmark on fixture: 1,739 msecs (273 detected 100 final)
doublestars. SEP: 66.464s (11207 detected 100 final)
sunflower    SEP: 5.623s (680 detected 100 final)
fullframe:   SEP: 4.180s (427 detected 100 final)
crop frame   SEP: 0.081s (3 detected 3 final)

RPi4 4Gb Raspbian WITH SPEEDUP

QBenchmark on fixture: 818ms (273 detected 100 final)
doublestars. SEP: 6.187s (11207 detected 100 final)
sunflower    SEP: 2.424s (680 detected 100 final)
fullframe:   SEP: 2.363s (427 detected 100 final)
crop frame   SEP: 0.083s (3 detected 3 final)

Also, note the large difference between a fast macbookpro and an RPi4 4Gb!

Addressing your comment for line 789 of fitsview.cpp.
> I disagree : it was like this before, but small stars ellipses become offset because of rounding issues on the pixel center.

See the images in this google photos album: https://photos.app.goo.gl/CHgXVZBREA19LVgN8

Note that using the sep 'width' that is calculated is very inconsistent. It is slightly larger than twice the HFR (I don't mind that, I could go with that if you strongly preferred it) but occasionally it is way too large. It is not at all hard to find these aberrations. If you wanted wider circles, I could use 3xHFR or something like that, but the width seems wrong too often. I guess the width somehow includes the entire flux above background, which can be fooled by fluctuations in background or something like that.

Re you comment on fitssepdetector.cpp line 135:
> What if 0.8 of the source set means 0 sources? Is the remaining algorithm safe? Shouldn't we discard sources based on their distance in sigma units instead? (that value may not be available, sure)

I guess I was planning on waiting on Rob's investigations to get "the perfect" parameters (e.g. 20% vs 2sigma). Top 20% is very rough, but it tended to do the job. As Pit observed in the forums, the HFR should be similar for all stars.
The code seems safe as long you can safely sort a 0-length container, which should be ok. If you prefer, we could put this here: if (catalog->nobj == 0) goto exit;

Re more test fixtures:
> I would also like you to add more FITS fixtures captured with the CCD Simulator or real pictures, with various FWHM values (use the focuser simulator) to verify your improvements.

I can add a few other fits files here with tests. One of my subs is 30Mb, seems too much for this? Don't have any defocussed ones saved, perhaps I'll shoot some tonight (and crop them).

murveit updated this revision to Diff 79982.Apr 13 2020, 7:12 AM

Added tests at 3 different HFRs

Golden values for number of stars detected and hfr for 3 images
processed by the SEP algorithm.
They are 2x2 bined focus images from an autofocus run of an ASI1600
at different stages of focus.

The only concert I have right now is the file size. We would be adding ~21MB worth of data to the repo, which is cloned and updated by numerous users and developers alike. Is there a way to cut this down? either by sampling each file to be < 1MB, or perhaps using 1/2 test files instead of 3? This would lessen the strain on the hosting services.

murveit updated this revision to Diff 80232.Apr 15 2020, 9:06 PM

Reduced the disk footprint of the test fixtures.

Reduced the disk size by cropping 2 of the 3 new text fixtures. Now 13.3Mb (was 23.3Mb).
Had to adjust the golden values for number of stars detected and resulting HFRs.

mutlaqja accepted this revision.Apr 15 2020, 10:53 PM
This revision is now accepted and ready to land.Apr 15 2020, 10:53 PM

Looks pretty good overall. Only suggestion I have is that I did find a way to get the saturation level that you might be interested in. I don't know that it is 100% foolproof, but see what you think:

https://github.com/rlancaste/sexysolver-tester/blob/master/sexysolver.cpp. Look here around line 284

murveit updated this revision to Diff 80261.Apr 16 2020, 6:17 AM

Modified saturation value if the data type is byte.

Rob, I actually disagree that you get the accurate saturation value. E.g. your technique wouldn't work for signed shorts or signed bytes or floating point where the range is 0 to 1.0, or for that matter, cameras where Indi would put out 0-4K values, etc. In the end, I think we'll need a user-supplied input for this, but all that, I think, should get worked out when you complete your Sextractor parameter integration, and this could be one of the parameters that we offer the user the ability to modify. Until then, I believe this will be good enough. It shouldn't do any harm, and it might improve the situation over the current system. I did, though, modify my code to check to see if it was a byte type, and if so, change the saturation threshold to 250.

About my concerns on rendering red circles, my point was that when you use a center and a radius to draw an ellipse, the center is set to a pixel. Therefore you risk to offset your ellipses when you should be using a subpixel as center. Agreed, this is only visible on very small stars, but it was annoying for me at the time, and made me incorrectly think this could be the reason my focus wasn't good or my solver was failing (newbie thoughts of course, but very real).

But when I look closely at your pictures, it seems the circles are offset to the right and bottom in both cases. I'm not referring to the width, I'm really talking about the center. Note that the pencil I was using had a width of 2, so it might be the reason? A pencil graphics code drawing stuff with all lines widths extended only to the right and bottom? That's entirely possible (and quite hateful).

murveit updated this revision to Diff 80263.Apr 16 2020, 8:04 AM

Adjust the center position of circles drawn around stars by half a pixel.

Eric, I apologize but I don't fully understand your suggestion. I haven't changed it from an ellipse to a circle. It was a circle before, and I've kept it a circle. What I was trying to do is remove the noise from using the poor width estimate as a radius and replace it with double the HFR, which is much more consistent, as my images show. I didn't intentionally change where the center was, so if the center is off now, it was likely off before, e.g. note in the images I sent, the images on the right, offsets are noticable on some stars, is the "before" picture, that doesn't have my changes. Note, one thing I did change, though, is now I give Qt a float for the coordinates, whereas previous it was truncated to an int. Do you think that had an adverse affect? If you see an error in my code for centering these circles, please let me know and I'd be happy to fix it.

That said, I did notice that https://sextractor.readthedocs.io/en/latest/Param.html says: "in SExtractor the center of the first image pixel has coordinates (1.0,1.0)"
and https://doc.qt.io/archives/qt-4.8/coordsys.html shows that coordinates 1,1 in Qt would be one half pixel down and to the right of that.
So, I've modified this code and subtracted 0.5 from the x and y center coordinates. You can see a screen shot of the NGC2239 image with that change
(and with the # of stars to detect temporarily set to 1000, and the top-20% removal disabled). Perhaps that's better. Some stars are a little offset, some are not.
Perhaps it's just SEP?

All that said, I'd prefer to not delay this PR for that, so I can remove this if you still feel strongly about it, and we can discuss later. Please let me know your preference.

Forgot to add the URL to the example image in my last comment.
It's https://photos.app.goo.gl/tG5yheZyfxHWoJ788
and it's also at the bottom of the album with all the images:
https://photos.app.goo.gl/CHgXVZBREA19LVgN8

"Rob, I actually disagree that you get the accurate saturation value. E.g. your technique wouldn't work for signed shorts or signed bytes or floating point where the range is 0 to 1.0, or for that matter, cameras where Indi would put out 0-4K values, etc. In the end, I think we'll need a user-supplied input for this, but all that, I think, should get worked out when you complete your Sextractor parameter integration, and this could be one of the parameters that we offer the user the ability to modify. Until then, I believe this will be good enough. It shouldn't do any harm, and it might improve the situation over the current system. I did, though, modify my code to check to see if it was a byte type, and if so, change the saturation threshold to 250."

Hy you are of course correct on this, and I have not fully finished it yet as you pointed out. One possible thought I have on how to fix this is to either divide by 2 or raise to one less power of two for signed data types, but the only way I can think to determine whether it is signed or not is to just have an if statement or case switch statement based on the types that should be signed. I am not sure that we can determine what saturated means for a float or double image, but we might not have to because their max value is incredibly high. I did already make a user supplied value, basically a threshold percentage of saturation they would like to remove, I think this will be easier for end users to work with, assuming that I can get the data types worked out.

TallFurryMan accepted this revision.Apr 16 2020, 8:41 PM

I will certainly not block a differential on an issue with source centers which we do not use afterwards :) Sorry for forgetting the "Accept Revision" flag.

My point was really simply that the code was previously drawing canvas ellipses with drawEllipses() using the top-left and width/height values, while the new one is using the center point and radius values.
When you use the center with the second prototype, you risk offsetting your ellipse/circle by one (eventually scaled) pixel, while using an enclosing rect is a better approach in my opinion when the picture has a low resolution or is scaled.

I also observe offsets in your images, I wonder if it might only be the width of the pen that is used by the renderer. Using a width of 1 instead of 2 might resolve the issue.
Your solution adding 0.5 to the center position seems improper to me: there's no reason we should do that.

lancaster accepted this revision.Apr 16 2020, 9:07 PM
mutlaqja accepted this revision.Apr 17 2020, 8:57 AM
This revision was automatically updated to reflect the committed changes.