Agent-based models have been used to examine many different aspects of tumor biology, allowing researchers to explore how tumors develop both temporally and spatially. However, a computational model is only as good as its parameters, which need to be properly set to reflect actual biological behavior. In order to determine these parameters, model simulations need to be compared to experimental data. This poses a unique challenge for agent-based models of tumors, as these models account for complex spatial dynamics and thus a comparison to image data is needed to properly estimate parameters. Here, we present a method for performing a direct comparison between model simulations and tumor images by using representation learning. Representation learning is the use of neural networks to project complex inputs to low-dimensional points, with points that are closer together being more similar to each other than points that are farther away. We use this to project model simulations and tumor images to low-dimensional space and then take the distance between the two, using this distance as an objective function for parameter estimation. We show how we process images and model simulations into a comparable format, how we train the neural network to learn low-dimensional representations of the two, and how it can then be applied as an objective function. We demonstrate its success both on model-generated data, showing that it can be used to accurately estimate model parameters that were used to generate a test simulation, and then on actual tumor images. Ultimately, we provide a novel method for performing a quantitative comparison between agent-based model simulations and tumor images that allows for parameter estimation.