2.2.1.

Population data: 2008 census

The main population data that were used is the 2008 census database. The file, which is in .shp format, contained the administrative divisions of the country, the division of certain large agglomerations into enumerated areas (EA), and the population sizes that were associated with these different divisions. Data on population size (per household) and housing characteristics were used, which had been collected during various surveys and polls that were conducted after the 2008 census.

2.2.2.

VHR satellite images

The choice of search areas for VHR images corresponding to the 2008 census was based on the density of buildings/population, following verification of field data (photographs and videos), on Google Earth and on OpenStreetMap. Four subareas of interest were considered given the availability of good quality VHR images, which were provided at that time only by QuickBird (Fig. 1). This image (65 cm QuickBird with pan-sharpening) was acquired on January 16, 2009, which was close to the census year (hereafter, referred to as the 2008 image).

The recent VHR image that was used to estimate the population and guide future censuses is a Pleiades image (50 cm with pan-sharpening) that was acquired on January 4, 2021. Figure 2 shows an example of built-up areas on the two types of VHR images that were used (QuickBird, 2009; Pleiades, 2021) in a residential area of Greater Khartoum.

Fig. 2

Extracts of VHR images: (a) QuickBird from January 16, 2008 and (b) Pleiades from January 4, 2021.

2.2.3.

Taxonomy and constitution of the dataset

To our knowledge, no semantically rich, annotated building detection dataset exists that takes into account the realities of our study area. Therefore, we constituted our own dataset for training the neural network.

Exploration of the 2008 census data allowed us to establish a taxonomy of residential geo-object classes (houses) according to the number of inhabitants per unit (Table 1). The data included a “household questionnaire” (housing characteristics), an examination of the VHR images, the state of the art, the images and photographs that were available on Google Earth, as well as field data (photographs taken on site).

Table 1

Description of different classes of residential geo-objects (houses).

Figure 3 illustrates the key to interpreting the taxonomy of different classes that were selected. Image preprocessing consisted of an enhanced QuickBird image in the RGB composite; this step was performed the match the RGB format of the datasets that were used in the CNN. Images were then sliced and converted to an unsigned 8-bit format, which was compatible with the libraries and faster R-CNN model that was used.

Fig. 3

Interpretation key for annotating house classes: (a) single-story house, (b) two-story house, (c) multistory house, (d) semicommercial house, (e) nonresidential house, (f) tall building, and (g) other.

Manual annotation of residential geo-object classes, which were clipped to 10 QuickBird image areas of interest using a GIS tool (QGis), resulted in a .shp file with ground-truthed bounding boxes that mark terrain coordinates, while specifying class and probability of the object. The images of these areas of interest have been cut into thumbnails of $300\times 300\text{pixels}$, with 50% overlap between them. This overlap is applied so that information is not lost should a building be cut between two images. Due to the very limited quantity of annotated data, it was necessary to perform data augmentation (flips, and 90 deg and 270 deg rotations), which is a practice that is widely used with CNNs.²⁹ A total of 556 $300\times 300\text{pixel}$ thumbnails, covering 2413 annotations, were used for training (80% of the data) and validation (20% of the data) of the faster R-CNN model. They are distributed according to Table 2, which summarizes the class structure of the dataset. A large imbalance characterized the distribution of classes with an imbalance ratio (number of majority annotations/number of minority annotations) of 93.6. The distribution has been assigned in such a way that a thumbnail image and its versions resulting from the augmentation are found in the same dataset. The situation where an image seen in training is found during validation is thus avoided. Figure 4 shows examples of annotation of different classes of residential geo-objects.

Table 2

Distribution of annotations according to the class of residential geo-objects.

Fig. 4

Different types of annotations of residential geo-objects on VHR thumbnails.

2.3.1.

Detection of residential geo-objects by deep learning

The irregular nature of buildings in the study area and the difficulty of recognizing house classes in VHR images make the annotation task difficult. Judicious photo interpretation was necessary to provide a good dataset on which the detection model can be trained. The faster R-CNN model,³⁰ which has been shown to perform well in computer vision applications,^22,31 was used. With the generalization capabilities of pretrained models, for example, PASCALVOC-2012³² in the case of faster R-CNN, it is possible to commence training from prior knowledge rather than starting from scratch.²⁹ Our residential geo-object detection approach is an application of this transfer learning technique.³³

The operation of faster R-CNN is based on three main components. (a) The first component is a pretrained model (i.e., backbone), the goal of which is to extract deep features from the input image. The convolutional neural network (ResNet50) was selected for the task after various tests of network depths; this particular network is 50-layers deep. (b) The second component is a region proposal network,³¹ which generates bounding boxes in the input image, followed by a bounding box pooling layer that projects the attributes extracted by the backbone to a vector and, finally, the application of classification and regression operators. (c) The third component is a classification operator that is combined with a regression operator to estimate the probability of each bounding box as belonging to a class (thanks to the classification operator) and to refine its final positioning (thanks to the regression operator). The cross-entropy loss function in Ref. 31 was used. Since our class distribution was not very homogeneous, the technique that was used by Ref. 34 is adopted, which consists of weighting the class loss function. The function was estimated as the total number of annotations per thumbnail, over all classes, divided by the total number of annotations per thumbnail of each class; 1 is subtracted from the result. We obtained the following weights: 1 (background); 0.41 (class 1: single-story house); 0.68 (class 2: two-story house); 0.96 (class 3: multistory residential house); 0.96 (class 4: semicommercial/semiresidential house); 0.99 (class 5: nonresidential house); 0.99 (class 6: tall building); and 0.98 (class 7: other). As is the case with hyper-parameters, we used the Adam stochastic gradient optimizer³⁵ with a learning rate of 0.0001 and a batch size of 10 thumbnails. Windows 11 and the Torchvision library from PyTorch were used as the computing framework. The computer that was used was equipped with an Intel Core i7 CPU and a Nvidia RTX GPU with 16 GB of memory.

2.3.2.

Detection evaluation

The evaluation of the performance of our detection approach is achieved using mean average precision (mAP) in a multiclass detection system.³⁶ AP is a widely used metric in many object detection applications, where its values are calculated over recall values ranging from zero to one. Recall that “recall” is calculated as the number of true positives, divided by the sum of true positives plus false negatives. Computation of mAP thus involves important submetrics, such as recall, precision, intersection over union (IoU), the precision recall curve (PRC), and a confusion matrix.³⁶ The GitHub page by Rafael Padilla ( https://github.com/rafaelpadilla/Object-Detection-Metrics) should be consulted regarding details of this metric and its concepts. Given the complexity of the built environment in the Greater Khartoum area, an $\mathrm{IoU}\ge 50\%$ was used in our study. Furthermore, PRC evaluates classification tasks better with unbalanced data,³⁷ which justifies its selection for our study.

2.3.3.

Generalization of the model in the 2008 image

Once the model is trained, inferential detection is performed over the entire 2008 image. This allows characterization of the buildings across the whole study area and uses these to establish the “house–population” relationship from the 2008 census.

2.3.4.

Characterization of residential geo-objects from the 2021 image

The faster R-CNN model that was trained on the 2008 QuickBird image is used to detect houses in the 2021 image, after fine-tuning. Thus, annotations were performed on the 2021 images. In total, 2861 annotations were made, with proportions per class relatively similar to those of 2008. Unsurprisingly, they were made in the same areas as the 2008 image. The increase in the number of annotations on the 2021 image is explained primarily by the appearance of new buildings between 2008 and 2021 (see Fig. 2).

2.3.5.

Population estimates

The 2008 census data provide a population count at the enumerated area (EA) level. These data were used to establish a “house–population” relationship. An average population factor is determined by residential geo-object class using the population data by EA from the 2008 Census according to the following equation

For example, if $E{A}_1$ contains 100 residential geo-objects of class 1, 50 of class 2, and 0 of the other classes, and $E{A}_1$ has a population size of 600 according to the census, a possible solution might be that ${\mu }_1=5$ and ${\mu }_2=2$ ($600=100\times 5+50\times 2$). Choice of a likely solution is guided by the dominance of each house class in the EAs. Thus, all EAs are traversed to estimate $\mu$ that was found for each class while taking into account information from informal household surveys. We assumed that the household size per house class (${\mu }_i$) did not change between 2008 and 2021, having found no evidence for this.

3.1.1.

On the 2008 image

The faster R-CNN network was trained over 20 epochs. Figure 5 shows the mAP curve as a function of each epoch (i.e., one cycle or pass through the full training dataset). Following an increase in mAP during the first 10 epochs, the pattern achieved a stable plateau. The final mAP that was recorded at the end of the 20th epoch was about 79% during training and about 99% during validation. Loss reached 0.12 by the end of 20 epochs of training. Note that the constitution of the training and validation datasets was done randomly on the annotated thumbnails. However, the validation set contains almost instances of the majority classes (1 et 2) on which the model has very well performed in training. Since these are the instances that the model sees more in validation, the mAP in validation becomes higher than the mAP in training.

Fig. 5

Evolution of precision during training and validation, and loss in training for the faster R-CNN model.

Given that training curves remain stable after a number of epochs reflects nonrepresentativeness in the dataset,³⁸ a nonrepresentative dataset may not capture statistical characteristics relative to another dataset that is drawn from the same domain, such as between training and validation data. This can usually happen if the number of samples in one dataset is too small compared with another dataset. Two common cases could be observed: (a) the training dataset is relatively unrepresentative; or (b) the validation dataset is relatively unrepresentative. In our particular case, it is the validation dataset that is poorly represented.

The precision–recall curve (Fig. 6) shows that six of the seven residential geo-object classes are well detected by the model, with precision ranging from 84% to 100% (Table 3). Detection of Class 5 (“nonresidential house”) seems to be difficult, with an $\mathrm{AP}=25\%$. Note that this is the least represented class in the dataset ($<1\%$). In general, the area under the precision–recall curve shows a large area between the curves and the baseline. This translates into perfect performance of the model and, therefore, appropriate functioning of the classifier, with a mAP of 79.46% for all seven classes.

Fig. 6

Precision of detection for different house classes.

Table 3

Average precision (AP) among classes of residential geo-objects.

In comparing the AP of each class to the distribution of class annotations (Table 2), the model clearly detects classes that appear more frequently in the training dataset and has difficulties for classes that appear less frequently. The detection performance of our approach is limited by the class imbalance.

Figure 7 shows the confusion matrix, which is a convenient and intuitive way of evaluating a model with respect to its detection performance on individual classes. In the case of a multiclass classification, a false negative in one class according to the prediction will be a false positive of the other classes, i.e., false negatives are found in the rows (observed) that are associated with each class, and false positives are in the columns (predicted) that are associated with each class.

Fig. 7

Detection confusion matrix of the seven residential geo-object classes.

Classes 1, 2, 3, 4, and 5 are better detected than 6 or 7, with success rates exceeding 60%. Yet, it should be noted that some confusion in detection persists among classes. The most important ones are: (a) class 7 (“other”) versus class 1 (“single-story house”), with 54% of class 7 confused with class 1; (b) class 6 (“tall building”) versus class 7 (“other”), at 43%; and (c) between class 4 (“semicommercial/residential house”) versus class 1, at 24%. Some classes have a detection rate below 60%. Other less important confusions are observed between several classes, reflecting the inability of the model to correctly detect certain types of buildings. Figure 8 shows examples of (a) and (b) good detections and (c) and (d) bad detections.

Fig. 8

Examples illustrating detections with faster-R-CNN on the 2008 QuickBird image: (a) and (b) good detection. (c) and (d) bad detections.

In general, large area geo-objects (e.g., nonresidential houses, schools, services, and mosques, among others) are not well detected. These geo-objects are often a collection of other classes. Occasionally, the model detects these classes individually in this case, rather than the set forming a distinct geo-object [circled in Fig. 8(d)]. Another explanation could be that the area of these geo-objects is larger than the size of the patches (300 × 300 pixels) that were cut to train the model. The corresponding annotations are cut into several patches, thereby preventing the model from properly learning how to recognize these residential geo-objects.

Major confusion between class 1 and 7 (54%) could be explained by the very definition of the latter class. Indeed, class 7 is designated “other,” given that it consists of a set of buildings for which identification was problematic during the annotation process. Class 1 geo-objects have characteristics in common with structures annotated in this seventh class; hence, the confusion that arose during detection. We also noted “empty fences” throughout the study area, i.e., parcels enclosed by a perimeter fence where the interior was empty, giving the appearance of a residential geo-object. The model detects these objects as constructed (either class 1 or class 7), but occasionally as background. This increases the confusion between classes 1 and 7.

3.1.2.

On the 2021 image

Fine-tuning of the model on the Pleiades image resulted in detections of residential geo-objects in 2021 on all four subareas of the study site. Figure 9 illustrates detections, along with a zoomed inset of a residential area as an example.

Fig. 9

Detection of residential geo-objects on Pleiades images from 2021 (on the ensemble of certain RoIs and extract of RoI).

3.2.1.

Population estimated for 2008

Analysis of the “house–population” relationship was performed on 87 EAs. Figure 10 shows the distribution of the numbers of each class detected by each EA and the average population/house ratio. The most dominant class among EAs is class 1, followed by classes 7 and 2. The average population/house ratio across the AEs varies greatly, ranging from 1.4 (likely an error in the census database) to 19.1 (likely a very special case).

Fig. 10

Distribution of each class population/house ratio by EA.

We further note that variation in the average population/house ratio is related to the dominance of a class in an EA. The principle of the method for estimating this ratio is based on finding the EAs with a single dominant class and estimating its ratio, then looking for EAs with a second dominant class and evaluating its ratio, and so on.

The dominant classes that were detected are 1, 7, and 2, respectively. Their degree of dominance is related to variation in the population/house ratio. We note that for EAs with 90% dominance of class 1, the population/house ratio varies between 9 and 13. When class 7 is important (EAs with 35%), variation ranges between 4 and 9. When class 2 reaches 25% in the EAs, we observe a population/house ratio that varies between 4 and 7.

The different variations in the population/house ratio guided the choice of the $\mu$ factor (average population size per class), starting with the dominant classes and rejecting implausible extreme/erroneous values. For the other classes, estimation of $\mu$ is based on Eq. (1), while considering the data that were reported by field specialists. Thus, the $\mu$-factors per residential geo-object class are as shown in Table 4.

Table 4

Estimated mean sizes by residential geo-object class.

Applying these different average sizes to all houses that were detected across all EAs yields a good correlation between the total population of EAs and the total population of all houses in that EA, as shown in Fig. 11, with the exception of some EAs that were investigated.

Fig. 11

Correlation of total EA population and total population per house.

Analysis of Figs. 10 and 11 reveals three main sets (Table 5): (a) 6.9% of houses with $\mu >2$; (b) 87.36% of houses with $\mu$ between 4 and 14; and (c) 5.75% of houses with $\mu <15$. The set in case (b) corresponds to the range of population sizes per house that were generally reported in the field. The two possible outlier groups could likely be attributed to data errors, as discussed below.

Table 5

Variation in population/house ratios of EAs.

We note the EAs that were of concern are all found in the same zone for each set (Fig. 12). EAs with a population/house size $\text{ratio}<2$ are very close to an area where population data for some EAs are missing from the census database. Estimates for the population data that were captured in these EAs are likely erroneous (Fig. 12, in blue, EAs with missing data in the database and in black, EAs with probably erroneous data).

Fig. 12

Illustration explaining EAs with bad population data: (a) EAs with missing data and (b) EAs with probably erroneous data.

EAs with a population/house size $\text{ratio}>15$ originate from areas where division of EAs is often difficult. In general, several enumerators are placed in these areas for systematic surveying, with the risk of double counting. Population estimates in these EAs are often high compared to the average. Population sizes of 25,298 and 10,739 are observed in the EA of this zone [Fig. 12(b)].

3.2.2.

Population estimated for 2021

Applying the model to the 2021 images resulted in detections of residential geo-objects in 2021 across the study site. Some examples at the scale of an EA are shown in Figs. 13 and 14. Tables 6 and 7 show an estimate of population in 2021 in these AEs by applying the average population sizes by house class determined above. The same value of the $\mu$-factor (average population size per class) has been used (in the absence of a reliable update). The increase in population is therefore due to new residential developments that are revealed in the 2021 image.

Fig. 13

Example 1 of detection for 2021.

Fig. 14

Example 2 of detection for 2021.

Table 6

Example 1 of population estimation for an EA.

Table 7

Example 2 of population estimation for an EA.