SYSTEM AND METHOD FOR DETERMINING HIGH-TEMPERATURE REGION PIXELS ON A MAMMOGRAPHY IMAGE

20240415477

18/696949

September 22, 2022

A system for annotating mammography images of a subject using thermal images and mammography images of the subject using thermal and mammography images of the subject by (i) identifying a first breast region in the thermal image and a second breast region in the mammography image, (ii) identifying a block of pixels (Pt) with a high temperature region associated with a breast lesion within the first breast region, (iii) estimating a location (l) of the breast lesion corresponding to the identified block of pixels (Pt), (iv) determining a block of pixels (Pm) corresponding to the location (l) of the block of pixels (Pt) within the second breast region and (v) generating a report with an annotated mammography image with markings of a determined block of pixels (Pm) on the mammography image of the subject corresponding to the block of pixels (Pt) on the thermal image.

1. A system for annotating mammography images of a subject using thermal images and mammography images of the subject by determining a block of pixels on the mammography images corresponding to high temperature regions on the thermal images of the subject, the system comprising: a thermal imaging device that captures a thermal image of the subject; a mammography imaging device that captures the mammography image of the subject; and a processor that is configured to:  characterized in that, identify a first breast region in the thermal image of the subject; identify a second breast region in the mammography image of the subject; identify a block of pixels (Pt) with the high temperature region associated with a breast lesion within the first identified breast region; estimate a location (l) of the breast lesion corresponding to the identified block of pixels (Pt); determine, using a first machine learning model, a block of pixels (Pm) on the mammography image corresponding to the location (l) of the block of pixels (Pt) within the second breast region; and generate a report with an annotated mammography image with a marking of a determined block of pixels (Pm) on the mammography image of the subject corresponding to the block of pixels (Pt) associated with the high temperature regions on the thermal image to enable lesion identification on the mammography image of the subject.

2. The system of claim 1, wherein the thermal imaging device comprises an array of sensors that converts infrared energy into electrical signals on a per-pixel basis, wherein the array of sensors detects temperature values from the subject; and a specialized thermal processor that processes detected temperature values into pixels of a thermal image, wherein intensity values of the pixels correspond to the detected temperature values.

3. The system of claim 1, wherein the mammography imaging device comprises an X-ray tube that produces low energy X-rays; a plurality of filters that are placed in a path of X-ray beam to modify an X-ray spectrum that is projected on a body of the subject; a plurality of compression paddles that is attached to the body of the subject to compress a part of the body of the subject being exposed to the X-rays to obtain cross section density information; and a specialized mammogram processor that converts obtained cross section density information into pixels to generate a mammography image, wherein intensity values of the pixels correspond to the obtained cross section density information at per-pixel basis.

4. The system of claim 1, wherein the processor is configured to identify the block of pixels (Pt) with high temperature regions on the thermal image of the breast region of the subject by determining a first pixel region (m1) with a temperature Tpixel, where T2≤Tpixel≤T1, wherein T1, and T2 are temperature thresholds obtained from the temperature distribution of the thermal image of the subject.

5. The system of claim 1, wherein the processor is configured to identify the block of pixels (Pt) with the high temperature regions on the thermal image of the breast region of the subject by: determining the first pixel region (m1) with a temperature T1pixel, where T2≤T1pixel≤T1; determining a second pixel region (m2) with a temperature T2pixel, where T3≤T2pixel; and detecting a plurality of hotspot regions using the first pixel region (m1) and the second pixel region (m2) with AND or OR rules, wherein T1, T2 and T3 are temperature thresholds obtained from the temperature distribution of the thermal image of the subject.

6. The system of claim 1, wherein the processor is configured to obtain the plurality of high temperature regions on the first breast region of the subject as an input from the user to estimate the location (l) of the breast lesion corresponding to the block of pixels (Pt) in the thermal image of the subject.

7. The system of claim 1, wherein the processor is configured to identify block of pixels (Pt) with high temperature regions on the first breast region of the subject using a second machine learning model, wherein the second machine learning model is trained by providing a plurality of thermal images and corresponding annotated high temperature regions associated with different patients as training data.

8. The system of claim 1, wherein the first machine learning model identifies the block of pixels (Pm) on the mammography image corresponding to the location (l) on the thermography image by: obtaining a breast quadrant corresponding to the location (l); identifying a view of the mammography image; dividing the mammography image into different quadrants; and identifying the block of pixels (Pm) that lie within a obtained quadrant corresponding to location (l) of the breast lesion on the thermographic image.

9. The system (104) of claim 1, wherein the first machine learning model identifies the block of pixels (Pm) on the mammography image corresponding to the location (l) on the thermography image by: identify candidate block of pixels with high density in the mammography image; obtaining a nipple point (N) on the mammography image; calculating locations (lm1-n) of each candidate block of pixels with respect to a obtained nipple point (N); comparing the locations (lm1-n) of each of the candidate block of pixels with the location (l) of the breast lesion on the thermographic image; and selecting the block of pixels (Pm) among the candidate block of pixels by selecting a block of pixels corresponding to a nearest location (lmi) among the locations (lm1-n) that is close to the location (l) of the breast lesion on the thermographic image.

10. The system of claim 1, wherein the first machine learning model identifies the block of pixels (Pm) on the mammography image corresponding to the location (l) on the thermography image comprises: obtaining a clock position (θ) and a distance (r) corresponding to the location (l); dividing the mammography image into different sectors corresponding to different clock positions; and identifying the block of pixels (Pm) that lie within the sector corresponding to the clock position (θ) and distance (r) from the nipple region in the mammography image.

11. The system of claim 9, wherein the processor is configured to identify the block of pixels (Pm) as the high-density regions in the mammography image within the sector corresponding to the clock position (θ) and distance (r) from the nipple region in the mammography image.

12. The system of claim 10, wherein the processor is configured to identify the high-density regions on the mammography image of the subject using a third machine learning model, wherein the third machine learning model is trained by providing a plurality of mammography images and the corresponding annotated high-density lesions associated with different patients as training data.

13. The system of claim 1, wherein the report comprises at least one of annotated mammography image with markings of the determined block of pixels (Pm) in a different color as annotations on the mammography image; annotated mammography image with markings of the boundary of the determined block of pixels (Pm) in a different color as annotations on the mammography image; a text report that comprises quantitative parameters of the block of pixels (Pm) on the mammography image corresponding to the high temperature region on the thermal image.

14. The system of claim 12, wherein the markings comprise an annotation of the block of pixels (Pm) on the mammogram image corresponding to the high temperature region on the thermal image and a text annotation that comprises quantitative parameters of the block of pixels (Pm).

15. A method for annotating mammography images of a subject using thermal images and mammography images of the subject by determining a block of pixels on the mammography images corresponding to high temperature regions on the thermal images of the subject comprising, capturing a thermal image of the subject using a thermal imaging device; capturing the mammography image of the subject using a mammography imaging device; identifying a first breast region in the thermal image of the subject; identifying a second breast region in the mammography image of the subject; identifying a block of pixels (Pt) with the high temperature region associated with a breast lesion within the first identified breast region; estimating a location (l) of the breast lesion corresponding to the identified block of pixels (Pt); determining, using a first machine learning model, a block of pixels (Pm) on the mammography image corresponding to the location (l) of the block of pixels (Pt) within the second breast region; and generating a report with an annotated mammography image with a marking of a determined block of pixels (Pm) on the mammography image of the subject corresponding to the block of pixels (Pt) associated with the high temperature regions on the thermal image to enable lesion identification on the mammography image of the subject.

16. The method of claim 15, further comprises identifying the block of pixels (Pt) with high temperature regions on the thermal image of the breast region of the subject by determining a first pixel region (m1) with a temperature Tpixel, where T2≤Tpixel≤T1, wherein T1, and T2 are temperature thresholds obtained from the temperature distribution of the thermal image of the subject.

17. The method of claim 15, further comprises identifying the block of pixels (Pm) on the mammography image corresponding to the location (l) on the thermography image by: obtaining a breast quadrant corresponding to the location (l); identifying a view of the mammography image; dividing the mammography image into different quadrants; and identifying the block of pixels (Pm) that lie within a obtained quadrant corresponding to location (l) of the breast lesion on the thermographic image.

18. The method of claim 15, wherein the first machine learning model identifies the block of pixels (Pm) on the mammography image corresponding to the location (l) on the thermography image by: identify candidate block of pixels with high density in the mammography image; obtaining a nipple point (N) on the mammography image; calculating locations (lm1-n) of each candidate block of pixels with respect to a obtained nipple point (N); comparing the locations (lm1-n) of each of the candidate block of pixels with the location (l) of the breast lesion on the thermographic image; and selecting the block of pixels (Pm) among the candidate block of pixels by selecting a block of pixels corresponding to a nearest location (lmi) among the locations (lm1-n) that is close to the location (l) of the breast lesion on the thermographic image.

19. The method of claim 15, wherein the first machine learning model identifies the block of pixels (Pm) on the mammography image corresponding to the location (l) on the thermography image comprises: obtaining a clock position (θ) and a distance (r) corresponding to the location (l); dividing the mammography image into different sectors corresponding to different clock positions; and identifying the block of pixels (Pm) that lie within the sector corresponding to the clock position (θ) and distance (r) from the nipple region in the mammography image.

20. A non-transitory computer-readable storage medium storing the one or more sequence of instructions, which when executed by one or more processors, causes to perform a method for annotating mammography images of a subject using thermal images and mammography images of the subject by determining a block of pixels on the mammography images corresponding to high temperature regions on the thermal images of the subject comprising, capturing a thermal image of the subject using a thermal imaging device; capturing the mammography image of the subject using a mammography imaging device; identifying a first breast region in the thermal image of the subject; identifying a second breast region in the mammography image of the subject; identifying a block of pixels (Pt) with the high temperature region associated with a breast lesion within the first identified breast region; estimating a location (l) of the breast lesion corresponding to the identified block of pixels (Pt); determining, using a first machine learning model, a block of pixels (Pm) on the mammography image corresponding to the location (l) of the block of pixels (Pt) within the second breast region; and generating a report with an annotated mammography image with a marking of a determined block of pixels (Pm) on the mammography image of the subject corresponding to the block of pixels (Pt) associated with the high temperature regions on the thermal image to enable lesion identification on the mammography image of the subject.

61; 16

edspap.20240415477