Research - ML based Embryo Implantation Forecast

Sep 27

Written By Rohit M

Introduction

Successful In-vitro fertilization (IVF) hinges on the precise timing of embryo transfer into the uterus, followed by its successful implantation. Uterus Peristalsis (UP) ias a critical factor in this implantation process, marked by the rhythmic contractions and relaxations of the smooth uterine muscles. Extensive literature reveals the pivotal role of these contractions (UC) in fertility, with their significance peaking during the early lumenal phase and diminishing rapidly thereafter. Consequently, the accurate measurement and potential manipulation of these contractions could offer a substantial enhancement in the overall success rates of fertility treatments.

Research Question

Is it possible to develop machine learning models capable of assessing the readiness of the uterus for implantation and estimating the time remaining until it's prepared?

Mouse Model of Uterus

This project is part of an NHS funded initiative with Prof. Arora of MSU. We used mice uterus to model UP. To study the uterine contractions and relaxation, we removed the uterus from mice that were in various stages of pregnancy. These uteruses were scanned with confocal microscope to image the status of the electrical activity at each position from oviduct to cervical end of the uterus. A typical scan looks as shown in Figure 1. The x-axis shows the horn position in pixels and the Y axis shows the frames. The scans were done during the 3rd after breeding and the day after implantation as shown in Figure 2.

Figure 1: Scan of the uterus showing the electrical activity along the uterus wall

Figure 2: T2, T4 and T6 are scans taken 3 days after breeding. Post implantation on Day 4 is recorded as T4.

Method

Approach: This research uses mice uterus images to train the machine learning model. The approach to train the models involve 4 steps (Figure 3). The data is first augmented by padding and cropping the images. This not only increases the training set but also makes the training images uniform size. Then the data is split into 80, 10, 10 groups for training, validation and testing. The training data is used to train a Vision Transformer model to classify the images from one the four time periods during the implantation process, namely, Day 3: T2, T3, T4 and Day 4: T4. The best model is selected for validation and testing. Data allocated for validation and testing are used with the best model to classify the images.

**Figure 3**: Pipeline to classify images as one of four time periods during implantation

Training Architecture: Traditional image augmentation methods such as rotation, shifts, sheer, flips etc won’t work because the timing of the event and the horn positions need to be maintained constant. As a workaround, I used the following image preprocessing algorithm (Figure 4): Each image was padded to transform the rectangular shaped image to a square shaped image. The image was then cropped to a size of 224x224. The set of images was then transposed to linear sequences and passed to the vision transformer as inupt. A total of 60 test cycles were used to train the model and the best trained model was selected for subsequent classification.

Testing Architecture: For testing the model, I used the following steps (Figure 5): Each image for testing was first cropped into equal sized patches of size 224x224. Each patch was passed through the decoder part of vision transformer model created during the training phase and probability scores were generated for each patch. Probabilities were rank ordered across all the patches that make up one whole training image. Probabilities of the patches were compared with those of the trained model and the label of the image that had the highest match in probabilities was selected

Results

The training loss and validation loss are shown in Figures 6a and 6b. Training loss improved by 75% over 42 epochs and validation loss improved by 10% within 12 epochs before the process was terminated.

Figure 6a: Chart showing the training loss

Figure 6b: Chart showing the validation loss

Statistical Analysis
We analyzed the results using z test, as shown below, and found that the ML model was better than randomly guessing when the image was scanned.

Saliency Maps

The saliency map for each of the scanned image of the uterus (left) is shown below. The brightness of a pixel in the saliency map is directly proportional to its saliency.

Future Directions

To enhance the model's accuracy, a strategic approach involves hyperparameter tuning, with particular emphasis on optimizing the learning rate, Adam optimization, Beta 1, and Beta 2. Moreover, automating the model to generate the optimal image at each timepoint, rather than depending on saliency maps for interpretation, could prove advantageous. Additionally, scaling up GPU capacity holds the potential to significantly reduce the current estimated 7-hour training time. Lastly, streamlining the model to a more compact size would render it deployable on low-capacity machines.

Conclusion

Successful IVF requires well-times transfer of an embryo into the uterus. This timing is dictated by the frequency of smooth muscle contractions within the uterine walls. The primary objective of this project was to gauge the readiness for implantation in a mouse uterus using a machine learning model trained on an extensive dataset of scanned mouse uteri at different pregnancy stages. Our findings demonstrate that our model achieved accurate predictions for 60% of the test images, with an overall accuracy rate of 64% for images drawn from a randomized sample. The model’s accuracy has the potential for further enhancement through the fine-tuning and optimization of various hyperparameters.

Rohit M