To create a concrete example, we\u2019ll use the scheme used by James\nMcCaffrey in his June 2019 Test Run column entitled Simplified\nNaive Bayes Classification Using C#<\/a><\/em>. \nOne can imagine that we are pawn brokers in McCaffrey\u2019s universe. People frequently come in hawking jewelry and,\ngiven that we run a pawn shop, we should expect that some of our clientele are\nless than trustworthy. We want build a\nmodel that allows us to classify a gemstone as being real or fake based on its\ncolor, size, and style of the cut.<\/p>\n\n\n\n These three attributes of the gemstone will be factors used\nto make the prediction and they are typically arranged in a list or array that\nis euphemistically called a vector (it is only euphemistically so as these list\ndon\u2019t obey the accepted definitions for a vector space). The gemstone vector will have 3 dimensions\nfor color, size, and style. Each\nattribute has various realizations as shown in this figure<\/p>\n\n\n\n To develop our model we first have to pool together what we\nknow based on the gemstones we\u2019ve seen. \nFor example, if a kind-hearted woman who had fallen on hard times came\nin with a small, twisted aqua-colored stone that we verified was authentic then\nwe would enter into our database the entry:<\/p>\n\n\n\n where the \u20181\u2019 means authentic or good. If some shady character, acting all tough came\nin with a small, blue, pointed stone that we reluctantly took and found out\nlater was fake we would amend our database to read:<\/p>\n\n\n\n where the \u20180\u2019 means fake or bad. Proceeding in this fashion, we produce a\ntraining set for our agent to gain experience with as it develops its own\ninternal mode. For this initial prototype,\nI used the 40 element training set provided by McCaffrey (of which the first\ntwo points are as shown above). <\/p>\n\n\n\n This kind of training is called supervised since we actually\nlabel each feature vector with the category into which it belongs. It is worth noting that there isn\u2019t a single Bayesian\nclassifier but rather a family of related algorithms. The basic concepts are the same but the\nparticular way in which the training set is characterized leads to better or\nworse performance based on context. In\nparticular, all NBCs assume that a given attribute is independent of the value\nof any other attribute.<\/p>\n\n\n\n Anyway, returning to McCaffrey\u2019s NBC, the structure of his\nalgorithm is most easily summarized in the following steps (the names of my\nPython routines to implement these steps are shown in parentheses):<\/p>\n\n\n\n The primary data structure is the python dictionary which\nbuilds up around each attribute discovered in the training set. Obviously, this limits the NBC to classifying\non known attributes. In other words, if\na ruby-colored gemstone came on the scene the agent\/model wouldn\u2019t know how to classify\nit. This situation would be the same for us manning the pawn shop when a person\nwho don\u2019t know whether to trust of not comes in with such a stone.<\/p>\n\n\n\n The code for each function is listed here: <\/p>\n\n\n\n Happily, the code reproduces the results of McCaffrey\u2019s original article but preliminary tests with more varied training sets have been disappointing.\u00a0<\/p>\n","protected":false},"excerpt":{"rendered":" As discussed in the last column, classification can be a tricky thing. Much of the machine learning buzz centers on classification problems. Typical examples include things like optical character recognition… Read more ><\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-1079","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"_links":{"self":[{"href":"https:\/\/aristotle2digital.blogwyrm.com\/index.php?rest_route=\/wp\/v2\/posts\/1079","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/aristotle2digital.blogwyrm.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/aristotle2digital.blogwyrm.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/aristotle2digital.blogwyrm.com\/index.php?rest_route=\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/aristotle2digital.blogwyrm.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=1079"}],"version-history":[{"count":0,"href":"https:\/\/aristotle2digital.blogwyrm.com\/index.php?rest_route=\/wp\/v2\/posts\/1079\/revisions"}],"wp:attachment":[{"href":"https:\/\/aristotle2digital.blogwyrm.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=1079"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/aristotle2digital.blogwyrm.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=1079"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/aristotle2digital.blogwyrm.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=1079"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}![\"\"](\"http:\/\/aristotle2digital.blogwyrm.com\/wp-content\/uploads\/2020\/08\/Gemstones.png\")
<\/figure>\n\n\n\nAqua,Small,Twisted,1<\/pre>\n\n\n\n
Aqua,Small,Twisted,1
Blue,Small,Pointed,0 <\/pre>\n\n\n\ndef find_distinct_values(df,attributes_lst):\n distinct_values = {}\n for attribute in attributes_lst:\n distinct_values[attribute] = set(df[attribute])\n\n return distinct_values\n<\/pre>\n<\/div>\n\n\n\ndef calculate_Laplace_smoothed_marginals(df,distinct_values):\n #initialize the marginals\n marginals = {}\n for attribute_type in distinct_values:\n for attribute in distinct_values[attribute_type]:\n #initializing to [1,1] implements Laplace smoothing\n marginals[attribute] = np.array([1,1]) \n \n for attribute_type in distinct_values:\n for attribute, authenticity in zip(df[attribute_type],df['authenticity']):\n marginals[attribute][authenticity] += 1\n \n return marginals \n<\/pre>\n<\/div>\n\n\n\ndef characterize_data_set(df):\n fake_label = 0\n true_label = 1\n summary = {}\n authenticity_data = df['authenticity']\n fake_counts = len(np.where(authenticity_data==fake_label)[0])\n true_counts = len(np.where(authenticity_data==true_label)[0])\n \n summary['num samples'] = fake_counts + true_counts\n summary['num fake'] = fake_counts\n summary['num true'] = true_counts\n \n return summary\n<\/pre>\n<\/div>\n\n\n\ndef calculate_evidence(distinct_values,smoothed_marginals,summary,sample_values):\n fake_label = 0\n true_label = 1\n num_attributes = len(distinct_values)\n \n prob_fake = summary['num fake']\/summary['num samples']\n prob_true = summary['num true']\/summary['num samples']\n smoothed_num_fake = summary['num fake'] + num_attributes\n smoothed_num_true = summary['num true'] + num_attributes\n \n sample_evidence_fake = 1\n for attribute in sample_values:\n sample_evidence_fake *= smoothed_marginals[attribute][fake_label]\/smoothed_num_fake\n sample_evidence_fake *= prob_fake\n \n sample_evidence_true = 1\n for attribute in sample_values:\n sample_evidence_true *= smoothed_marginals[attribute][true_label]\/smoothed_num_true\n sample_evidence_true *= prob_true\n \n normalization = sample_evidence_fake + sample_evidence_true\n \n return sample_evidence_fake\/normalization, sample_evidence_true\/normalization\n<\/pre>\n<\/div>\n\n\n\n