Many types of vegetables share similarities in color, texture, and shape, and even have different names in different countries. From production to delivery, several steps such as picking and sorting are still performed manually. This makes it challenging for customers to distinguish between similar vegetables at the market. The reliance on manual labor through many stages of vegetable production and consumption significantly hampers the commercialization of vegetable products.

To address this issue, implementing automation in the processes of picking, sorting, and labeling through a vegetable image classifier is essential, as it would save both time and money. In contemporary agriculture, fundamental research focuses on classification and detection because there are various kinds of vegetables that many people are unfamiliar with. This is why I decided to choose vegetable as the focus of the clustering and classification for this assignment.

I searched Kaggle for an appropriate dataset to work with. I want to work with as big a data set and I am allowed to (in this case being 200 images), so Kaggle seemed the efficient choice in finding all the labelled images I needed in classified folders. I was lucky to find the paper “DCNN-Based Vegetable Image Classification Using Transfer Learning: A Comparative Study” by Asif Uz Zaman Asif, Mohammed Israk Ahmed, and Shahriyar Mahmud Mamun. The dataset they used for their research was exactly what I needed, and their large dataset provided all the testing images I required. In fact, I had to remove most of the dataset to stay within the 200 images limit.

Part 1

For the section of the assignment, I collected 20 images of 10 different vegetables, all in one folder to makeup a dataset of 200 images. The vegetables were tomato, radish, pumpkin, potato, cucumber, cauliflower, carrot, capsicum, cabbage, and broccoli.

• Fig 1.1 Marked Inception v3 labelled image grid*

Fig 1.1 is image grid created by the inception v3 algorithm that I’ve marked over. I think it did a relatively decent job at organizing the vegetables. On the left are most of the green leafy and the right is populated by all the other colors, but mainly orange vegetables. With this being the cause veg like cabbage, broccoli end up on the left and carrots and potatoes end up on the right. Also visible is a diagonal strip of white veg, consisting of cauliflower and radishes, running across the columns. We can also make out the algorithm has also considered the shape and texture of vegetables. The cylindrical/cone shaped veg (like carrots and radishes) are concentrated in the bottom right of the grid.

The more knobbly, cloud shaped veg (like broccoli and cauliflower) are concentrated along the left end of the grid. We can also see a variance in plurality in the grid. The images in the bottom right have many more vegetables in each image compared other images on the grid. This is most probably due to similar shots taken of the carrots, radishes, and cucumbers. To sum up, most vegetables look like they’ve been well within their groups, except for capsicums which are more sparsely spread out near the middle of the grid.

Fig 1.2 Painters image grid

Fig 1.2 is the resulting image grid when using the Painters algorithm. I think this algorithm does a better job at grouping colors together but not the contents of the images (in this case being vegetables). What I mean by this is, this grid has a clear group of orange and red vegetable in the top right corner of the grid, but it doesn’t do a good job of distinguishing between capsicums and carrots in that corner.

Also, like with the inception v3 generated grid, there seems to a spread of white vegetable (radishes and cauliflower) across columns. Like Inception v3, the Painter algorithm looks to be having a hard time grouping the capsicum together, perhaps due to their different colors. One major difference is that in Fig 1.2, there is no clear association or relation between broccoli and cauliflower, whereas in Fig 1.1, we saw the two vegetables groups next to each other.

Fig 1.3 SqueezeNet image grid

We see, a similar job done by the SqueezeNet algorithm. I was surprised to see most of the capsicums grouped together in the top right, which was better than the previous two algorithms had grouped them. However, SqueezeNet has performed considerably worse with the other vegetable groupings. One of which is cucumbers, which is spread throughout the grid, from the very top to the very bottom.

Fig 1.4 top half hierarchical clustering by Inception v3

Fig 1.5 bottom half hierarchical clustering by Inception v3

Fig 1.6 section of bottom half hierarchical clustering by Inception v3

Fig 1.7 images of section of bottom half hierarchical clustering by Inception v3

I am very curious as to how the algorithm “sees” these vegetables clustered together in Fig 1.6 to put them together. Perhaps it is there roughly spherical shapes, but the carrots violate this relation. Perhaps it is the red/orange hues of the vegetables that bring them together, but the green tomatoes seem to be outliers in this case. Majority of the images are of tomatoes, which the algorithm seems to groups together based on their shapes. I can understand how the algorithm might have mistaken the smooth curved exterior of the capsicums to be that of tomatoes. The carrots however look completely different. Perhaps the algorithm is seeing something in the circular stumps of the carrots, but I cannot say I am confident about that.

It took me long to organise and name all the images for the dataset, and I was only working with 200 samples. How much more manual work would be needed when working with datasets in the millions or billions? The chapter from “Distant Viewing: Computational Exploration of Digital Images” discusses the application of computer vision algorithms to digital images, which is referred to as “distant viewing.” This technique involves using computational methods to analyze large collections of images by creating structured annotations that capture essential information within these images. These annotations are used to explore and interpret visual messages across a collection, enabling a new kind of visual analysis that exceeds human capabilities in terms of scale and detail.

The chapter emphasizes that while computer vision can process images quickly and on a large scale, the annotations it produces are influenced by the cultural, social, and technical contexts in which the algorithms were developed. Therefore, distant viewing is not just a technical method but also involves critical engagement with the ways images make meaning and how they are processed computationally. Distant viewing is presented as an iterative, exploratory process, mirroring traditional methods used for smaller image collections but enhanced by the speed and scalability of computer vision. This approach allows for deep insights into visual cultures, powered by the ability to analyze vast numbers of images rapidly and detect patterns that may not be visible to the human eye.

Part 2

For this section of the assignment, I stored all the 200 images of vegetables into their corresponding folders. I ended up with 10 folders of 20 images each.

Fig 1.8 confusion matrix for inception v3

Fig 1.9 false positive for radish which is actually a broccoli

Fig 2.1 false positive for radish which is actually tomatoes

Fig 2.2 false positive for potato which is actually carrot

Fig 2.3 false positive for potato which is actually cucumber

All things considered, inception v3 did is good job with classifying the vegetables according to the confusion matrix in Fig 1.8. Fig 1.9 shows a false positive for radish which is actually a broccoli. Perhaps this is because this is a very close image of the light green stem of the broccoli which could be confused for the long cylindrical body of a radish. In Fig 2.1, the algorithm registered a false positive for a radish which is actually tomatoes. I am not sure how Inception could have possibly decided this was a radish. None of the images of radishes in the dataset resemble Fig 2.1, so I am completely stumped here.

As for Fig 2.2, where the algorithm concluded a false positive for potato which is actually carrots, perhaps it was because of the white background. Eight of the potato images in the dataset also have white backgrounds. In the case of Fig 2.3, where Inception v3 saw a false positive for potato which is actually a cucumber, it might have been because of the hand holding the cucumber. Three of the images from the potato folder have a hand holding a potato as well.

Fig 2.4 confusion matric for Painter

Fig 2.5 confusion matric for SqueezeNet

Fig 2.4 and Fig 2.5 also show the confusion matrices of other algorithms, like Painter and SqueezeNet respectfully. They have performed more poorly when compared to Inspection v3.

Project Summary

Unproject Plan (by Ahsen and Yerk)

For this assignment, I was excited to analyse a book series I had recently completed reading; the Farseer Trilogy, by acclaimed author Robin Hobb. The series invites readers into a captivating world where intrigue, magic, and political machinations intertwine. Set in the fictional realm of the Six Duchies, this epic fantasy saga follows the life of FitzChivalry Farseer, a royal bastard with a destiny intertwined with the fate of his kingdom. z As he navigates the complex web of courtly intrigue and battles against forces threatening the realm, Fitz discovers his innate talent for the ancient and mysterious art of the Skill, a form of telepathic communication, and the enigmatic Wit, a bond with animals. With richly drawn characters, immersive world-building, and a narrative that balances intimate personal struggles with grand-scale conflicts, the Farseer Trilogy is a spellbinding journey into a world where loyalty, betrayal, and sacrifice shape the destiny of nations.

In working on this series, I hope to uncover details I might have missed when reading the book.

Author

Author Margaret Astrid Lindholm Ogden

Robin Hobb, the pseudonym of Margaret Astrid Lindholm Ogden, is a revered figure in the realm of fantasy literature, celebrated for her extraordinary storytelling prowess and profound world-building skills. Born in 1952 in California, USA, Hobb’s journey as a writer began with her early works published under her birth name. However, it was under the guise of Robin Hobb that she truly flourished in the fantasy genre, distinguishing herself as a masterful creator of richly detailed worlds and deeply nuanced characters.

Analysis

Assassin’s Apprentice

Assassin’s Apprentice cover — book 1 of the Farseer Trilogy

Word cloud of Assassin’s Apprentice

The story follows Fitz, the illegitimate son of Prince Chivalry Farseer, who is taken in by the royal family of the Six Duchies. Despite the shame of his birth, Fitz is trained as a royal assassin and diplomat by master Chade. The above word cloud displays the 125 most frequently used words in the book. From this, we can look at the most prominent characters in Fitz’s life, like Burrich, Chade, Verity, Regal, Galen, and Shrewd, who are his guardian, mentor, uncle, uncle, instructor and King respectively.

Burrich being the most mentioned (428 times), is Fitz’s primary guardian and who Fitz interacts with the most during the series. This makes since Fitz is six years of age in the beginning of the book, and needs looking after. Burrich works in the stable at castle Buckkeep, which are both alluded to in the word cloud, with the mention of ‘Buckkeep’ and ‘horses’.

Loom tool visualization of Assassin’s Apprentice

The interactive chart shows the frequency of each word distributed throughout the book. Using it we can deduct major events in Fitz’s life, and his everchanging relationship dynamics. I set the pre-set to “terms whose distributions vary the most”, to filter out the data I am more interested in. Also, it is usually changes in particular frequencies that suggest events unfolding in the book.

By following the yellow line, we see Chade’s introduction in the book as Fitz’s mentor. In his lessons learning to be a royal assassin for the monarchy, he latches on to Chade as parental figure. In the same section of the chart that Chade’s frequency peaks, Burrich’s frequency dips (green line), indicating a shift in Fitz’s relation to his guardian. This is exactly what happens in the book as during this time, Fitz is angry and scared of Burrich after he takes his dog away and believes killed, and so he avoids Burrich.

In the middle of the story we also see the narrow peak of ‘Galen’ (indicated by a red line). This indicates the beginning of Fitz’s tortured tutelage under Galen in the Skill, which is a Telepathic kind of magic in this world. We can also see the climax of the book near the end represented by the peak of ‘Regal’, Fitz’s uncle and the main villain of the series (indicated by light green line).

Royal Assassin

Royal Assassin cover — book 2 of the Farseer Trilogy

Word cloud of Royal Assassin

Fitz forms a bond with a wolf named Nighteyes and navigates a romantic relationship with a maid named Molly, all while concealing his role as an assassin and his telepathic abilities. As the kingdom faces continued attacks from the Red-Ship raiders, Prince Verity seeks a solution through the use of the Skill, enlisting Fitz’s help in the war effort. Despite their efforts, the war escalates, prompting Verity to embark on a quest for mythical beings known as Elderlings.

With Fitz’s entering his teenage years in this series, we see new characters enter his life, reflected in the word cloud above. Molly, his romantic interest, Kettricken, Prince Verity’s bethroded, Patience, his stepmother, and Fool, his friend. We also see the magics of this world, Skill and Wit, take a stronger hold in this book. We get glimpses into Fitz’s life in this book with words like ‘boat’, ‘ship’ and ‘guard’ which hint at Fitz sailing into war, as a soldier, against the Outisland’s Red Ship raiders.

Assassin’s Fate

Assassin’s Quest cover — book 3 of the Farseer Trilogy

Word cloud of Assassin’s Quest

This is the longest book of the series, and covers the most content. After half recovering from trauma and seizures from the previous books, Fitz discovers that Regal has usurped the throne and moved the capital. Adopting a new identity, he sets out to assassinate Regal. After failing, he is bound by a Skill command to find Verity, who is attempting to awaken stone dragons to combat the raiders. Verity succeeds but sacrifices his humanity in the process. Verity destroys the raiders, and Kettricken ascends to the throne. Fitz decides to live as an outcast with Nighteyes, while Verity and the stone dragons protect the realm.

Again, with age with see Fitz meet new people that stick through the story, like ‘Starling’ and ‘Kettle’ and further deepen existing relations from previous books like his Wit bond with ‘Nighteyes’ and friendship with ‘Kettricken’. The word cloud also hints at the more fantastical elements of the epic fantasy, like ‘stone dragon’. A large section of the book is Fitz’s journey through the Six Duchies, past the Mountain Kingdom and beyond, which can be inferred from the appearance of landmarks like the Skill ‘road’ and ‘mountain’.

Loom tool visualization of Assassin’s Quest

Burrich’s line (reprented by navy blue) is highest at the start and stays low for the rest of the book, indicating that Fitz and Burrich part ways, which is what happens as this is when Fitz sets his mind on assassinating, now King Regal. We also see a step dip in the frequency of ‘Nighteyes’ (represented by grass green), Fitz’s wolf companion, in the first half of the chart. In the story, this is when Nighteyes leaves Fitz to live amongst fellow a pack of wolves to explore his wildness. The chart also shows when Nighteyes comes back to Fitz to save him, indicated by the rise in frequency.

We also see the appearance of new character like Starling (hazel line) and Kettle (grey-blue line) in the first half of the book, that stay with Fitz through his journey. We see Kettle’s line drop near the end of the book because she dies. There are also huge red and blue peaks in the middle of the book, representing ‘Fool’ and ‘Kettricken’ respectively, who are characters from the previous books. They also join Fitz on his journey to find King-in-waiting Verity. The peak in ‘Verity’ (pink line) near the end of the book shows that Fitz and his group did finally find him. At the very end of the book in a peak of ‘dragon’ (grey line), indicating the climax of the book and trilogy.

Conclusion

The visualization charts help pinpoint changes in the story and characters more precisely that what would normally be possible on a linear read. The “On the Way to Computational Thinking” says that “this way of seeing made possible by computation helps train the capacity to see effective solutions to research interests articulated through computation and formal analysis”. Almost like a bird’s eye view of the whole text, these tools help confirm certain insights from a different angle.

In reading the “Data modeling and Use” chapter from the “The Digital Humanities Coursebook” we dive deeper into how these databases work. They employ “parameterization (counting) and tokenization (what can be defined as a discrete unit) to produce quantitative or statistical information. Data may be qualitative as well as quantita-tive, and gathered with subjective criteria, but for purposes of processing, the data must be discrete, distinct, and unambiguous”. Making the data machine-readable allows analysis, repurposing, and manipulation of data/texts/files in systematic ways. Voyant tools uses this structured text to create its visually intuitive designs in order to better understand the text.

I found the Harvard Art Museum website easy to use and explore. Each art piece in their collection has its own pag to showcase and store its information. These pages take a minimalistic approach, with its black & white color scheme and uniform navigation. The website begins with the piece’s name and image, and is then followed by numerous titbits of information regarding the piece. This description includes, but is not limited to, its “identification and creation”, “physical description”, “acquisition and right”, “provenence” and “public history”. The website displays the details of individual pieces in an easily digestible manner.

Poet Sosei Hōshi of the printed book of “Thirty-Six Immortal Poets” on HAM website

However, there are limitations to what can be gleaned solely from the interface. Researchers and scholars may require access to raw data for more in-depth analysis and research. A CSV file provides structured data that can be imported into analytical tools or programming languages for statistical analysis, data visualization, and computational research. Further, the spreadsheet format of a CSV file allows users to perform custom queries and filters on the data to extract specific subsets of artworks-based combinations of attributes, which is not nearly possible to the same extent on the HAM website. Museums also constantly change their catalogue of art pieces, but the accurate data logging of their inventory is paramount. Updating such details is much simpler and quicker in a csv file, where documentation is standard, rather than websites which are often specialised for the piece it is housing.

In conclusion, while museum websites is great in providing access to collections information for the public, a CSV file is better suited towards data analysis, interoperability, and data preservation. By making structured data available in standard formats, museums can help researchers, scholars, and developers to explore, analyze, and interpret collections data in meaningful ways.

Part 2

The All_Culture_information csv file revealed that the vast majority of pieces in the Harvard Art Museum’s collection have European or American origins, with East Asians also making up a significant chunk. It is hard to find pieces of African, Arab, Central American or South American roots. While I expected there to be a sizable difference in these populations, I did not expect it to be this large.

Could it be because European and American art has been better documented, studied? These demographics of art are more often supported by wealthy patrons, institutions, and art markets. This longstanding tradition of artistic production and patronage has contributed to the proliferation and preservation of European and American art collections in museums. Or perhaps, in their effort for repatriation, the Harvard Art Museum has returned many of the non-American and non-European artifacts back to their countries of origins, thereby dwindling their collection?

Another possible explanation could be that HAM is only interested in pieces of particular roots, as museums often prioritize collecting art and artifacts that align with their areas of expertise and research interests. HAM may find it challenging to acquire and contextualize works outside their sphere. Donors often have specific preferences or connections to certain types of art, which can influence the direction and focus of museum acquisitions. In this case, donors may be more inclined to support the acquisition of European or American art due to personal interests, cultural affiliations, or perceived market value. The HAM also gets more visitors from America or Europe, which may incentivise HAM to prioritize the display of American and European art to enhance visitor engagement with local pieces.

Architectural Relief with Chamunda

Since I am from India, I decided to choose Indian culture to investigate on the Harvard_API_All_objects notebook. The most viewed item is ‘Architectural Relief with Chamunda’, viewed 1914 times. I do not know much about the piece itself, but I know that Chamunda is a Hindu deity, often associated with death, destruction, and fierce aspects of Devi, the Mother Goddess. Depictions of Chamunda can be found in various forms of Hindu art and sculpture, especially in regions where Devi worship is prevalent. Chamunda is typically portrayed with a fearsome appearance, often depicted with a skeletal form, adorned with skulls and wearing a garland made of decapitated heads. She is usually depicted standing on a corpse or a demon, symbolizing her role as a destroyer of evil forces.

I am not surprised that a piece inspired by this famous and polarizing Hindu Goddess has garnered the most views, as she is a popular subject to depict. Her images can be found in temples, shrines, and sacred spaces dedicated to Devi and other deities. The deity embodies the paradoxical nature of the Divine Mother in Hinduism, encompassing both fierce and compassionate aspects. Through her depiction in Hindu art, Chamunda serves as a powerful symbol of protection, transformation, and the eternal cycle of life and death.

The least viewed pieces were ‘Head of Animal Figurine (with snout), from Sari Dheri’ and ‘Head of Animal Figurine (with pointed ears), from Sari Dheri’, each with 3 views. Most of the pieces from Sari Dheri are unpopular, having views in the single digits or low double digits. It refers to the archaeological artifacts site of Sari Dheri, an ancient settlement located in present-day Pakistan. Sari Dheri is known for its rich archaeological remains dating back to the Indus Valley Civilization, one of the world’s earliest urban cultures. Animal figurines are common among the artifacts discovered at various Indus Valley Civilization sites. These figurines depict a variety of animals, including cattle, buffalo, dogs, elephants, and mythical creatures. They were likely used for religious, ritual, or decorative purposes. Perhaps their unpopularity stems from the pieces’ lack of detail, with time chipping away at it. Undoubtably being some of the oldest pieces in the Harvard Art Museum, their lustre has been inevitably lost, and their history with it.

Part 3

chart of cultures

In this section, I’ve chosen three distinct cultures: Korean, Indian, and Egyptian, as depicted above. These selections offer diverse cultural backgrounds, promising vastly contrasting word clouds. My aim was to curate cultures that possess comparable representation within the Harvard Art Museum collection. Each of these civilizations brings forth unique artistic traditions and historical narratives, enriching our understanding of human creativity and expression.

chart of accession year data of pieces from cultures

Above it is the accession year data and the time series bar chart. Most of the pieces obtained by the Harvard Art Museum seems to be in large bulk accessions. There never seems to be a steady flow-in of artifacts from any of the three cultures, but is instead large drops of pieces added to the collection. The museum also looks to have taken a more ready interest in the three cultures post 1970.

word cloud for Korean pieces

Stop words: “english”, “nan”, “of”, “the”, “a”, “from”, “with”, “i”, “are”, “for”, “it”, “and”, “to”, “by”, “that”, “on”, “Korean”, “in”, “an”, “as”, “at”

Above is the word cloud for artifacts of Korean culture. With “Sherd” and “Bowl” prominently featured, ceramics likely dominate the collection, possibly including decorative pieces, suggested by “floral” and “decor.” The appearance of “rice” and “cake” in the cloud is unexpected. Could these depict paintings or representations of culinary culture?

word cloud for Indian pieces

Stop words: “english”, “nan”, “of”, “the”, “a”, “from”, “with”, “i”, “are”, “for”, “it”, “and”, “to”, “by”, “that”, “on”, “Korean”, “in”, “an”, “as”, “at”

Above is the word cloud for artifacts of Indian culture. The presence of “manuscript” and “scripture” leads me to believe many of the artifacts were parchment or vellum documents. I was interested to see two distinct sub cultures from the Indian subcontinent; “Kota” and “Rajput”. The Kotas are an ethnic group indigenous to the Nilgiri mountain range in Tamil Nadu. Their religion and culture revolve around the smithy. The Rajput dynasty dominated northern India in the 7th century.

word cloud for Egyptian pieces

Stop words: “english”, “nan”, “of”, “the”, “a”, “from”, “with”, “i”, “are”, “for”, “it”, “and”, “to”, “by”, “that”, “on”, “Korean”, “in”, “an”, “as”, “at”

Above is the word cloud for artifacts of Egyptian culture. Like with Korean culture, “sherd” is prominent in the word cloud and likely also hints to ceramic artifacts dominating the collection. I was interested to see a mix of Islamic and ancient Egyptian concepts in the word cloud. Words like “Qur’an”, which is the Holy book of Islam and “Sura” which are chapters in the Qur’an, are alongside words like “Horus”, who is a sun god, and “Isis”, goddess of healing and magic, both from ancient Egyptian mythology. These concepts give us a window into the diverse understandings of the divine by the Egyptian people through millennia.

Remember, not all data exists, and even if it does, it might not have all the details you need. So, when you’re crafting your research question, make sure it’s clear and specific enough to be answered. Think about who you’re studying, what you’re studying, where it’s happening, when it’s happening, and how much data you need. These details will help you narrow down your search. To find data, look everywhere—from government agencies like NOAA, who gather lots of environmental data, to nonprofits like the OECD, private companies like Nielsen, and even academic repositories. When you use data in your work, always cite it properly. Just like you would with a book or an article, give credit to the people who created the data, mention when it was published, and provide a link if you can. If you’re having trouble finding data, try using a repository finder. These handy tools can help you locate open data sets based on what you’re studying and where you’re studying it. In the end, acquiring data is a crucial part of any research project. It might not always be easy, but with patience and perseverance, you’ll get there. In summary, finding the right data is like solving a puzzle. It takes time, effort, and sometimes a little luck. But by following these simple steps, you’ll be well on your way to uncovering the insights and answers you seek. Happy researching!